Jenkins build is back to normal : Atlas-master-UnitTests #2185

[Apache Jenkins Server]

See 

Re: Review Request 68358: Dependency upgrades

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68358/#review207305
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 15, 2018, 4:42 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68358/
> ---
> 
> (Updated Aug. 15, 2018, 4:42 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Upgrading to latest stable (non-breaking) dependencies
> 
> 
> Diffs
> -
> 
>   pom.xml f4686bf83 
>   webapp/pom.xml ab86a78e1 
> 
> 
> Diff: https://reviews.apache.org/r/68358/diff/1/
> 
> 
> Testing
> ---
> 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/570/console
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

[jira] [Commented] (ATLAS-2822) Provide option whether to delete propagated classification on entity delete during add classification

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580725#comment-16580725
 ] 

ASF subversion and git services commented on ATLAS-2822:


Commit bc093ca893eea6a84fceba1588fa48842cbe69e5 in atlas's branch 
refs/heads/master from [~sarath.ku...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=bc093ca ]

ATLAS-2822: Provide option whether to delete propagated classification on 
entity delete during add classification


> Provide option whether to delete propagated classification on entity delete 
> during add classification
> -
>
> Key: ATLAS-2822
> URL: https://issues.apache.org/jira/browse/ATLAS-2822
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
>
> By default, when an entity is deleted - its associated classifications which 
> have been propagated to downstream entities will be retained.
> This JIRA provides a boolean option when adding a new classification to an 
> entity - *"Remove Propagations on Entity Delete"*
> When this flag is set to :
> *TRUE* - Propagated classifications are removed during entity delete
> *FALSE* - Propagated classifications are retained during entity delete



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (ATLAS-2822) Provide option whether to delete propagated classification on entity delete during add classification

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580726#comment-16580726
 ] 

ASF subversion and git services commented on ATLAS-2822:


Commit c245049a7378887d1771a3ad8cd4898546ff6004 in atlas's branch 
refs/heads/branch-1.0 from [~sarath.ku...@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=c245049 ]

ATLAS-2822: Provide option whether to delete propagated classification on 
entity delete during add classification

(cherry picked from commit bc093ca893eea6a84fceba1588fa48842cbe69e5)


> Provide option whether to delete propagated classification on entity delete 
> during add classification
> -
>
> Key: ATLAS-2822
> URL: https://issues.apache.org/jira/browse/ATLAS-2822
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
>
> By default, when an entity is deleted - its associated classifications which 
> have been propagated to downstream entities will be retained.
> This JIRA provides a boolean option when adding a new classification to an 
> entity - *"Remove Propagations on Entity Delete"*
> When this flag is set to :
> *TRUE* - Propagated classifications are removed during entity delete
> *FALSE* - Propagated classifications are retained during entity delete



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68352: ATLAS-2822: Provide option whether to delete propagated classification on entity delete during add classification

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68352/
---

(Updated Aug. 14, 2018, 10:12 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Bugs: ATLAS-2822
https://issues.apache.org/jira/browse/ATLAS-2822


Repository: atlas


Description
---

By default, when an entity is deleted - its associated classifications which 
have been propagated to downstream entities will be retained.

This JIRA provides a boolean option when adding a new classification to an 
entity - "Remove Propagations on Entity Delete"

When this flag is set to :

TRUE - Propagated classifications are removed during entity delete

FALSE - Propagated classifications are retained during entity delete


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java d63b376a4 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
23ff5d28c 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
6422399f9 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 5aa40271b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
 3668e4796 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 b07ca1dcc 
  
repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java
 ce62ec555 
  repository/src/test/resources/tag-propagation-data-1.zip 
2c605499f871c59a470fdfbe2892e02e1805c7cd 
  repository/src/test/resources/tag-propagation-data.zip 
336f4fa6cfb7f2b593ef56b55d0a7354c25fb3bc 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 d50db0625 


Diff: https://reviews.apache.org/r/68352/diff/3/

Changes: https://reviews.apache.org/r/68352/diff/2-3/


Testing
---

* Added UT to cover this case.

* Validated using REST Client.

* PreCommit: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/566/console


Thanks,

Sarath Subramanian

Review Request 68358: Dependency upgrades

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68358/
---

Review request for atlas.


Repository: atlas


Description
---

Upgrading to latest stable (non-breaking) dependencies


Diffs
-

  pom.xml f4686bf83 
  webapp/pom.xml ab86a78e1 


Diff: https://reviews.apache.org/r/68358/diff/1/


Testing
---

https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/570/console


Thanks,

Apoorv Naik

Build failed in Jenkins: Atlas-master-UnitTests #2184

[Apache Jenkins Server]

See 

--
[...truncated 204.49 KB...]
[INFO] Exclude: .svn/**
[INFO] Exclude: .git/**
[INFO] Exclude: .gitignore
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.twiki
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.log
[INFO] Exclude: **/target/**
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 125 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 125 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 15 resources
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-solr-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ atlas-webapp 
---
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:clean (clean) @ atlas-webapp ---
[WARNING] Keystore file 
'
 doesn't exist.
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:generateKeyPair (generateKeyPair) @ 
atlas-webapp ---
[WARNING] 
[WARNING] Warning:
[WARNING] The JKS keystore uses a proprietary format. It is recommended to 
migrate to PKCS12 which is an industry standard format using "keytool 
-importkeystore -srckeystore 

 -destkeystore 

 -deststoretype pkcs12".
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ atlas-webapp 
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 69 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 48 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 

Build failed in Jenkins: Atlas-1.0-UnitTests #784

[Apache Jenkins Server]

See 

--
[...truncated 209.80 KB...]
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 124 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 124 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 15 resources
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-solr-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ atlas-webapp 
---
[INFO] artifact net.minidev:json-smart: checking for updates from central
[INFO] artifact net.minidev:json-smart: checking for updates from 
hortonworks.repo
[INFO] artifact net.minidev:json-smart: checking for updates from 
apache.snapshots.repo
[INFO] artifact net.minidev:json-smart: checking for updates from apache-staging
[INFO] artifact net.minidev:json-smart: checking for updates from default
[INFO] artifact net.minidev:json-smart: checking for updates from 
java.net-Public
[INFO] artifact net.minidev:json-smart: checking for updates from 
repository.jboss.org-public
[INFO] artifact net.minidev:json-smart: checking for updates from typesafe
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:clean (clean) @ atlas-webapp ---
[WARNING] Keystore file 
'
 doesn't exist.
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:generateKeyPair (generateKeyPair) @ 
atlas-webapp ---
[WARNING] 
[WARNING] Warning:
[WARNING] The JKS keystore uses a proprietary format. It is recommended to 
migrate to PKCS12 which is an industry standard format using "keytool 
-importkeystore -srckeystore 

 -destkeystore 

 -deststoretype pkcs12".
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ atlas-webapp 
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 68 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 48 source files to 

[INFO] 

Re: Review Request 68352: ATLAS-2822: Provide option whether to delete propagated classification on entity delete during add classification

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68352/#review207289
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Lines 118 (patched)


do retrySleepTimeMillis and removePropagations have to be 'public' 
accessiable? Also, consider marking all 3 statics here as 'final'.


- Madhan Neethiraj


On Aug. 15, 2018, 12:05 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68352/
> ---
> 
> (Updated Aug. 15, 2018, 12:05 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2822
> https://issues.apache.org/jira/browse/ATLAS-2822
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> By default, when an entity is deleted - its associated classifications which 
> have been propagated to downstream entities will be retained.
> 
> This JIRA provides a boolean option when adding a new classification to an 
> entity - "Remove Propagations on Entity Delete"
> 
> When this flag is set to :
> 
> TRUE - Propagated classifications are removed during entity delete
> 
> FALSE - Propagated classifications are retained during entity delete
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java d63b376a4 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
> 23ff5d28c 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 6422399f9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  5aa40271b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  3668e4796 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  b07ca1dcc 
>   
> repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java
>  ce62ec555 
>   repository/src/test/resources/tag-propagation-data-1.zip 
> 2c605499f871c59a470fdfbe2892e02e1805c7cd 
>   repository/src/test/resources/tag-propagation-data.zip 
> 336f4fa6cfb7f2b593ef56b55d0a7354c25fb3bc 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  d50db0625 
> 
> 
> Diff: https://reviews.apache.org/r/68352/diff/2/
> 
> 
> Testing
> ---
> 
> * Added UT to cover this case.
> 
> * Validated using REST Client.
> 
> * PreCommit: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/566/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 68353: ATLAS-2823: updated hooks to support asynchronous notifications

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68353/
---

(Updated Aug. 15, 2018, 1:21 a.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, Nixon Rodrigues, Ramesh 
Mani, and Sarath Subramanian.


Changes
---

- updated for the review comment
- additional refactoring to move maxRetries configuration to AtlasHook


Bugs: ATLAS-2823
https://issues.apache.org/jira/browse/ATLAS-2823


Repository: atlas


Description
---

updated hooks to support asynchronous notifications


Diffs (updated)
-

  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
77177b49f 
  
addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
 09998cfd0 
  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
78f2e8304 
  addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java 
ac4960122 
  
addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
 17493f616 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java bf6a36c59 


Diff: https://reviews.apache.org/r/68353/diff/2/

Changes: https://reviews.apache.org/r/68353/diff/1-2/


Testing
---

Verified that pre-commit tests pass successfully - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/567/


Thanks,

Madhan Neethiraj

Re: Review Request 68353: ATLAS-2823: updated hooks to support asynchronous notifications

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68353/#review207283
---


Ship it!




Ship It!

- Sarath Subramanian


On Aug. 14, 2018, 5:19 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68353/
> ---
> 
> (Updated Aug. 14, 2018, 5:19 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Nixon Rodrigues, 
> Ramesh Mani, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2823
> https://issues.apache.org/jira/browse/ATLAS-2823
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> updated hooks to support asynchronous notifications
> 
> 
> Diffs
> -
> 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java
>  77177b49f 
>   notification/src/main/java/org/apache/atlas/hook/AtlasHook.java bf6a36c59 
> 
> 
> Diff: https://reviews.apache.org/r/68353/diff/1/
> 
> 
> Testing
> ---
> 
> Verified that pre-commit tests pass successfully - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/567/
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Review Request 68353: ATLAS-2823: updated hooks to support asynchronous notifications

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68353/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, Nixon Rodrigues, Ramesh 
Mani, and Sarath Subramanian.


Bugs: ATLAS-2823
https://issues.apache.org/jira/browse/ATLAS-2823


Repository: atlas


Description
---

updated hooks to support asynchronous notifications


Diffs
-

  
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java 
77177b49f 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java bf6a36c59 


Diff: https://reviews.apache.org/r/68353/diff/1/


Testing
---

Verified that pre-commit tests pass successfully - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/567/


Thanks,

Madhan Neethiraj

Re: Review Request 68312: AttributeDef Update: Support for Soft Reference

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68312/#review207275
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
Line 506 (original), 506 (patched)


Move creation of 'objectIds' list from line #506 to #511.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
Line 514 (original), 518 (patched)


objectId could be null here, due to line #527, #541 below. Please handle 
this case, just as in line #497 above.


- Madhan Neethiraj


On Aug. 14, 2018, 10:50 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68312/
> ---
> 
> (Updated Aug. 14, 2018, 10:50 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2813
> https://issues.apache.org/jira/browse/ATLAS-2813
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> - Add _options_ field to _AtlasAttributeDef_.
> - Implement support in _AtlasStructDefStore_.
> - Update _EntityGraphMapper_
> - Update _EntityGraphRetriever_
> - Supports array of objects and objects.
> 
> **Usage**
> See attached JSON.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AttributeMutationContext.java
>  b32c09255 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  381191986 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  81b553e6e 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/SoftReferenceTest.java
>  856f38347 
> 
> 
> Diff: https://reviews.apache.org/r/68312/diff/5/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Tests verify object, array, map cases.
> 
> **Functional tests**
> - Quick Start
> - Entity creation via REST APIs.
> - Entity creation via APIs.
> 
> 
> File Attachments
> 
> 
> rdbms-db.json
>   
> https://reviews.apache.org/media/uploaded/files/2018/08/13/ded0e80a-bc59-4dc4-ba74-d971c7af8aef__rdbms-db.json
> typesDef-soft-ref.json
>   
> https://reviews.apache.org/media/uploaded/files/2018/08/13/e7c79e63-d08b-41ff-97ef-85389a6d63e3__typesDef-soft-ref.json
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

[jira] [Created] (ATLAS-2823) Update hooks to enable asynchronous notifications

[Madhan Neethiraj (JIRA)]

Madhan Neethiraj created ATLAS-2823:
---

 Summary: Update hooks to enable asynchronous notifications
 Key: ATLAS-2823
 URL: https://issues.apache.org/jira/browse/ATLAS-2823
 Project: Atlas
  Issue Type: Bug
  Components: atlas-intg
Affects Versions: 1.0.0
Reporter: Madhan Neethiraj


With refactoring of hooks to use V2 style notifications (ATLAS-2491), the 
option to send notifications asynchronously was removed. Sending the 
notifications asynchronously is helpful, to avoid delays in host component; 
hence should be supported.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (ATLAS-2823) Update hooks to enable asynchronous notifications

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-2823:
---

Assignee: Madhan Neethiraj

> Update hooks to enable asynchronous notifications
> -
>
> Key: ATLAS-2823
> URL: https://issues.apache.org/jira/browse/ATLAS-2823
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Critical
>
> With refactoring of hooks to use V2 style notifications (ATLAS-2491), the 
> option to send notifications asynchronously was removed. Sending the 
> notifications asynchronously is helpful, to avoid delays in host component; 
> hence should be supported.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : Atlas-1.0-AllTests #767

[Apache Jenkins Server]

See 

Re: Review Request 68352: ATLAS-2822: Provide option whether to delete propagated classification on entity delete during add classification

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68352/#review207274
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
Lines 1315 (patched)


Consider reading the default value from configuration.


- Madhan Neethiraj


On Aug. 14, 2018, 10:32 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68352/
> ---
> 
> (Updated Aug. 14, 2018, 10:32 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2822
> https://issues.apache.org/jira/browse/ATLAS-2822
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> By default, when an entity is deleted - its associated classifications which 
> have been propagated to downstream entities will be retained.
> 
> This JIRA provides a boolean option when adding a new classification to an 
> entity - "Remove Propagations on Entity Delete"
> 
> When this flag is set to :
> 
> TRUE - Propagated classifications are removed during entity delete
> 
> FALSE - Propagated classifications are retained during entity delete
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java d63b376a4 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
> 23ff5d28c 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 6422399f9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  5aa40271b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  3668e4796 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  b07ca1dcc 
>   
> repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java
>  ce62ec555 
>   repository/src/test/resources/tag-propagation-data-1.zip 
> 2c605499f871c59a470fdfbe2892e02e1805c7cd 
>   repository/src/test/resources/tag-propagation-data.zip 
> 336f4fa6cfb7f2b593ef56b55d0a7354c25fb3bc 
> 
> 
> Diff: https://reviews.apache.org/r/68352/diff/1/
> 
> 
> Testing
> ---
> 
> * Added UT to cover this case.
> 
> * Validated using REST Client.
> 
> * PreCommit: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/566/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Jenkins build is back to normal : PreCommit-ATLAS-Build-Test #566-master-0001-ATLAS-2822-Provide-option-whether-to-delete-propagat.patch

[Apache Jenkins Server]

See 

Jenkins build is back to normal : Atlas-master-UnitTests #2182

[Apache Jenkins Server]

See 

Re: Review Request 68312: AttributeDef Update: Support for Soft Reference

[Ashutosh Mestry]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68312/
---

(Updated Aug. 14, 2018, 10:50 p.m.)


Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and 
Sarath Subramanian.


Changes
---

Updates include:
- Handled delete from list attribute.
- Handled delete from map attribute.
- Additional unit tests to verify delete and udpate in collection.


Bugs: ATLAS-2813
https://issues.apache.org/jira/browse/ATLAS-2813


Repository: atlas


Description
---

**Approach**
- Add _options_ field to _AtlasAttributeDef_.
- Implement support in _AtlasStructDefStore_.
- Update _EntityGraphMapper_
- Update _EntityGraphRetriever_
- Supports array of objects and objects.

**Usage**
See attached JSON.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AttributeMutationContext.java
 b32c09255 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 381191986 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 81b553e6e 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/SoftReferenceTest.java
 856f38347 


Diff: https://reviews.apache.org/r/68312/diff/5/

Changes: https://reviews.apache.org/r/68312/diff/4-5/


Testing
---

**Unit tests**
Tests verify object, array, map cases.

**Functional tests**
- Quick Start
- Entity creation via REST APIs.
- Entity creation via APIs.


File Attachments


rdbms-db.json
  
https://reviews.apache.org/media/uploaded/files/2018/08/13/ded0e80a-bc59-4dc4-ba74-d971c7af8aef__rdbms-db.json
typesDef-soft-ref.json
  
https://reviews.apache.org/media/uploaded/files/2018/08/13/e7c79e63-d08b-41ff-97ef-85389a6d63e3__typesDef-soft-ref.json


Thanks,

Ashutosh Mestry

Build failed in Jenkins: PreCommit-ATLAS-Build-Test #565-master-0001-Add-flag-to-toggle-removing-propagated-classificatio-MASTER.patch

[Apache Jenkins Server]

See 


--
[...truncated 403.30 KB...]
127.0.0.1 - - [14/Aug/2018:22:35:46 +] "POST 
//localhost:31000/api/atlas/types HTTP/1.1" 201 113 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:46 +] "POST 
//localhost:31000/api/atlas/entities/cdf5b553-3fcc-4c15-bade-89673476b9a1/traits
 HTTP/1.1" 201 72 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:47 +] "POST 
//localhost:31000/api/atlas/entities/cdf5b553-3fcc-4c15-bade-89673476b9a1/traits
 HTTP/1.1" 400 137 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:47 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 883 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:48 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2302 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:50 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 883 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:51 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2302 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:52 +] "POST 
//localhost:31000/api/atlas/entities/b054032f-d3b3-41fe-be3f-203b5d517147?property=description
 HTTP/1.1" 200 2162 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:52 +] "GET 
//localhost:31000/api/atlas/entities/b054032f-d3b3-41fe-be3f-203b5d517147 
HTTP/1.1" 200 2098 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:53 +] "POST 
//localhost:31000/api/atlas/entities/b054032f-d3b3-41fe-be3f-203b5d517147?property=invalid_property
 HTTP/1.1" 400 68 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:53 +] "GET 
//localhost:31000/api/atlas/entities/b054032f-d3b3-41fe-be3f-203b5d517147 
HTTP/1.1" 200 2098 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:53 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 883 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:53 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2302 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:55 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 881 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:55 +] "POST 
//localhost:31000/api/atlas/entities/d30bcb8b-1cbb-4612-817c-4957a3e7f75f?property=db
 HTTP/1.1" 200 820 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:55 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 883 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:56 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2302 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:58 +] "GET 
//localhost:31000/api/atlas/v2/types/typedef/name/PII_TraitHlpIH4gPFX HTTP/1.1" 
404 96 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:58 +] "POST 
//localhost:31000/api/atlas/types HTTP/1.1" 201 113 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:58 +] "POST 
//localhost:31000/api/atlas/entities/76b783af-a916-4798-9c1e-a6c20eb064f9/traits
 HTTP/1.1" 201 72 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:59 +] "GET 
//localhost:31000/api/atlas/entities/76b783af-a916-4798-9c1e-a6c20eb064f9/audit?count=100
 HTTP/1.1" 200 2855 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:59 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 883 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:35:59 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2302 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:01 +] "GET 
//localhost:31000/api/atlas/v2/types/typedef/name/PII_TraitRcOgjdI8nR HTTP/1.1" 
404 96 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:01 +] "POST 
//localhost:31000/api/atlas/types HTTP/1.1" 201 113 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:02 +] "POST 
//localhost:31000/api/atlas/entities/3886004a-337b-4f7b-97b7-bb39aab1ba6d/traits
 HTTP/1.1" 201 72 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:03 +] "GET 
//localhost:31000/api/atlas/entities/3886004a-337b-4f7b-97b7-bb39aab1ba6d 
HTTP/1.1" 200 2267 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:03 +] "POST 
//localhost:31000/api/atlas/entities/random/traits HTTP/1.1" 400 76 "-" 
"Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:03 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 883 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:03 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2302 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:06 +] "PUT 
//localhost:31000/api/atlas/entities HTTP/1.1" 200 1254 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:06 +] "GET 
//localhost:31000/api/atlas/entities/da290b96-ecd8-4f53-b5f6-1c103e2373c1 
HTTP/1.1" 200 3674 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:06 +] "POST 
//localhost:31000/api/atlas/entities HTTP/1.1" 201 2804 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:22:36:07 +] "POST 

Review Request 68352: ATLAS-2822: Provide option whether to delete propagated classification on entity delete during add classification

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68352/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Bugs: ATLAS-2822
https://issues.apache.org/jira/browse/ATLAS-2822


Repository: atlas


Description
---

By default, when an entity is deleted - its associated classifications which 
have been propagated to downstream entities will be retained.

This JIRA provides a boolean option when adding a new classification to an 
entity - "Remove Propagations on Entity Delete"

When this flag is set to :

TRUE - Propagated classifications are removed during entity delete

FALSE - Propagated classifications are retained during entity delete


Diffs
-

  common/src/main/java/org/apache/atlas/repository/Constants.java d63b376a4 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
23ff5d28c 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
6422399f9 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 5aa40271b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
 3668e4796 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 b07ca1dcc 
  
repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java
 ce62ec555 
  repository/src/test/resources/tag-propagation-data-1.zip 
2c605499f871c59a470fdfbe2892e02e1805c7cd 
  repository/src/test/resources/tag-propagation-data.zip 
336f4fa6cfb7f2b593ef56b55d0a7354c25fb3bc 


Diff: https://reviews.apache.org/r/68352/diff/1/


Testing
---

* Added UT to cover this case.

* Validated using REST Client.

* PreCommit: 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/565/console


Thanks,

Sarath Subramanian

[jira] [Updated] (ATLAS-2822) Provide option whether to delete propagated classification on entity delete during add classification

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2822:
--
Description: 
By default, when an entity is deleted - its associated classifications which 
have been propagated to downstream entities will be retained.

This JIRA provides a boolean option when adding a new classification to an 
entity - *"Remove Propagations on Entity Delete"*

When this flag is set to :

*TRUE* - Propagated classifications are removed during entity delete

*FALSE* - Propagated classifications are retained during entity delete

> Provide option whether to delete propagated classification on entity delete 
> during add classification
> -
>
> Key: ATLAS-2822
> URL: https://issues.apache.org/jira/browse/ATLAS-2822
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
>
> By default, when an entity is deleted - its associated classifications which 
> have been propagated to downstream entities will be retained.
> This JIRA provides a boolean option when adding a new classification to an 
> entity - *"Remove Propagations on Entity Delete"*
> When this flag is set to :
> *TRUE* - Propagated classifications are removed during entity delete
> *FALSE* - Propagated classifications are retained during entity delete



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2822) Provide option whether to delete propagated classification on entity delete during add classification

[Sarath Subramanian (JIRA)]

Sarath Subramanian created ATLAS-2822:
-

 Summary: Provide option whether to delete propagated 
classification on entity delete during add classification
 Key: ATLAS-2822
 URL: https://issues.apache.org/jira/browse/ATLAS-2822
 Project: Atlas
  Issue Type: New Feature
  Components:  atlas-core
Affects Versions: 1.0.0, 1.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.1.0, 1.0.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: Atlas-1.0-AllTests #766

[Apache Jenkins Server]

See 

--
[...truncated 308.39 KB...]
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 124 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 124 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 15 resources
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-solr-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ atlas-webapp 
---
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:clean (clean) @ atlas-webapp ---
[WARNING] Keystore file 
'
 doesn't exist.
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:generateKeyPair (generateKeyPair) @ 
atlas-webapp ---
[WARNING] 
[WARNING] Warning:
[WARNING] The JKS keystore uses a proprietary format. It is recommended to 
migrate to PKCS12 which is an industry standard format using "keytool 
-importkeystore -srckeystore 

 -destkeystore 

 -deststoretype pkcs12".
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ atlas-webapp 
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 68 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 48 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-webapp ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (unit-tests) @ atlas-webapp ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider

---
 T E S T S
---
Running org.apache.atlas.util.RestUtilsTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.812 sec - in 
org.apache.atlas.util.RestUtilsTest
Running org.apache.atlas.notification.AdaptiveWaiterTest
Tests run: 3, 

Build failed in Jenkins: Atlas-master-UnitTests #2181

[Apache Jenkins Server]

See 

--
[...truncated 204.28 KB...]
[INFO] Exclude: **/.checkstyle
[INFO] Exclude: **/*.txt
[INFO] Exclude: **/*.json
[INFO] Exclude: .pc/**
[INFO] Exclude: debian/**
[INFO] Exclude: .svn/**
[INFO] Exclude: .git/**
[INFO] Exclude: .gitignore
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.twiki
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.log
[INFO] Exclude: **/target/**
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 125 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 125 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 15 resources
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-solr-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ atlas-webapp 
---
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:clean (clean) @ atlas-webapp ---
[WARNING] Keystore file 
'
 doesn't exist.
[INFO] 
[INFO] --- keytool-maven-plugin:1.5:generateKeyPair (generateKeyPair) @ 
atlas-webapp ---
[WARNING] 
[WARNING] Warning:
[WARNING] The JKS keystore uses a proprietary format. It is recommended to 
migrate to PKCS12 which is an industry standard format using "keytool 
-importkeystore -srckeystore 

 -destkeystore 

 -deststoretype pkcs12".
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ atlas-webapp 
---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 69 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 48 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 

[jira] [Commented] (ATLAS-2724) UI enhancement for Avro schemas and other JSON-valued attributes

[Barbara Eckman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580339#comment-16580339
 ] 

Barbara Eckman commented on ATLAS-2724:
---

[~kevalbhatt] I just uploaded the patch file, developed by [~tstefanovicz]

> UI enhancement for Avro schemas and other JSON-valued attributes
> 
>
> Key: ATLAS-2724
> URL: https://issues.apache.org/jira/browse/ATLAS-2724
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.8-incubating
>Reporter: Barbara Eckman
>Assignee: Barbara Eckman
>Priority: Critical
> Fix For: 0.8.3
>
> Attachments: 0001-Add-pretty-printed-json-values-in-tables.patch
>
>
> Currently JSON-valued attributes are fully displayed in-line with other 
> attributes, not pretty-printed, cluttering the display.  To support a better 
> display, we can display JSON-valued attributes in a one-line box that can be 
> scrolled down, or fully expanded with a mouse click that pretty-prints the 
> JSON. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (ATLAS-2724) UI enhancement for Avro schemas and other JSON-valued attributes

[Barbara Eckman (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580339#comment-16580339
 ] 

Barbara Eckman edited comment on ATLAS-2724 at 8/14/18 7:49 PM:


[~kevalbhatt] I just uploaded the patch file, developed by [~tstefanovicz].  


was (Author: barbara):
[~kevalbhatt] I just uploaded the patch file, developed by [~tstefanovicz]

> UI enhancement for Avro schemas and other JSON-valued attributes
> 
>
> Key: ATLAS-2724
> URL: https://issues.apache.org/jira/browse/ATLAS-2724
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.8-incubating
>Reporter: Barbara Eckman
>Assignee: Barbara Eckman
>Priority: Critical
> Fix For: 0.8.3
>
> Attachments: 0001-Add-pretty-printed-json-values-in-tables.patch
>
>
> Currently JSON-valued attributes are fully displayed in-line with other 
> attributes, not pretty-printed, cluttering the display.  To support a better 
> display, we can display JSON-valued attributes in a one-line box that can be 
> scrolled down, or fully expanded with a mouse click that pretty-prints the 
> JSON. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2724) UI enhancement for Avro schemas and other JSON-valued attributes

[Barbara Eckman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Barbara Eckman updated ATLAS-2724:
--
Attachment: 0001-Add-pretty-printed-json-values-in-tables.patch

> UI enhancement for Avro schemas and other JSON-valued attributes
> 
>
> Key: ATLAS-2724
> URL: https://issues.apache.org/jira/browse/ATLAS-2724
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.8-incubating
>Reporter: Barbara Eckman
>Assignee: Barbara Eckman
>Priority: Critical
> Fix For: 0.8.3
>
> Attachments: 0001-Add-pretty-printed-json-values-in-tables.patch
>
>
> Currently JSON-valued attributes are fully displayed in-line with other 
> attributes, not pretty-printed, cluttering the display.  To support a better 
> display, we can display JSON-valued attributes in a one-line box that can be 
> scrolled down, or fully expanded with a mouse click that pretty-prints the 
> JSON. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2724) UI enhancement for Avro schemas and other JSON-valued attributes

[Barbara Eckman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Barbara Eckman updated ATLAS-2724:
--
Attachment: (was: atlas0_8UIChanges.tar)

> UI enhancement for Avro schemas and other JSON-valued attributes
> 
>
> Key: ATLAS-2724
> URL: https://issues.apache.org/jira/browse/ATLAS-2724
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.8-incubating
>Reporter: Barbara Eckman
>Assignee: Barbara Eckman
>Priority: Critical
> Fix For: 0.8.3
>
> Attachments: 0001-Add-pretty-printed-json-values-in-tables.patch
>
>
> Currently JSON-valued attributes are fully displayed in-line with other 
> attributes, not pretty-printed, cluttering the display.  To support a better 
> display, we can display JSON-valued attributes in a one-line box that can be 
> scrolled down, or fully expanded with a mouse click that pretty-prints the 
> JSON. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : Atlas-1.0-IntegrationTests #771

[Apache Jenkins Server]

See 

Build failed in Jenkins: Atlas-1.0-AllTests #765

[Apache Jenkins Server]

See 

--
[...truncated 99.11 KB...]
[INFO] Executing: /bin/sh -c cd 
' && 'git' 
'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: 

[INFO] Storing buildNumber: 705ce81ea2de7b7d2923b8d220575b20efca6bcf at 
timestamp: 1534272122847
[WARNING] Cannot get the branch information from the git repository: 
Detecting the current branch failed: fatal: ref HEAD is not a symbolic ref

[INFO] Executing: /bin/sh -c cd 
' && 'git' 
'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- apache-rat-plugin:0.12:check (rat-check) @ atlas-notification ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 61 implicit excludes (use -debug for more details).
[INFO] Exclude: **/antlr4/**
[INFO] Exclude: **/dependency-reduced-pom.xml
[INFO] Exclude: **/javax.script.ScriptEngineFactory
[INFO] Exclude: .reviewboardrc
[INFO] Exclude: 3party-licenses/**
[INFO] Exclude: **/.cache
[INFO] Exclude: **/.cache-main
[INFO] Exclude: **/.cache-tests
[INFO] Exclude: **/.checkstyle
[INFO] Exclude: **/*.txt
[INFO] Exclude: **/*.json
[INFO] Exclude: .pc/**
[INFO] Exclude: debian/**
[INFO] Exclude: .svn/**
[INFO] Exclude: .git/**
[INFO] Exclude: .gitignore
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.twiki
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.log
[INFO] Exclude: **/target/**
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 36 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 36 licenses.
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
atlas-notification ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-notification ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
atlas-notification ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 21 source files to 

[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-notification ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-notification ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 14 source files to 

[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 

[jira] [Commented] (ATLAS-2801) Create a separate tar file for atlas-server components excluding atlas hook related components

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580185#comment-16580185
 ] 

ASF subversion and git services commented on ATLAS-2801:


Commit f022990949e60e3b82e867bfc3845b3bb80859ca in atlas's branch 
refs/heads/master from [~ashutoshm]
[ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=f022990 ]

ATLAS-2801: Create a separate tar file for atlas-server.

Signed-off-by: Ashutosh Mestry 


> Create a separate tar file for atlas-server components excluding atlas hook 
> related components
> --
>
> Key: ATLAS-2801
> URL: https://issues.apache.org/jira/browse/ATLAS-2801
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: trunk
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Attachments: 
> 0001-ATLAS-2801-Create-a-separate-tar-file-for-atlas-serv.patch, 
> ATLAS-2801-Create-a-separate-tar-file-for-atlas-server-part-2.patch, 
> ATLAS-2801-Fix-for-tools-directory-absent-in-tar.gz.patch
>
>
> Create a separate tar file for atlas-server components excluding atlas hook 
> related components. Already we have separate tar files for each of the atlas 
> hooks and it's better to have a separate atlas-server tar file for better 
> management of the packages.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68270: Cluster Entity Updated after Successful Import if Replication Options are Available in Import Request

[Ashutosh Mestry]

> On Aug. 14, 2018, 6:34 a.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/clusterinfo/AtlasCluster.java
> > Lines 85 (patched)
> > 
> >
> > Use of stringified-AtlasObjectId as key doesn't sound good - since 
> > AtlasObjectId has guid/type/state/uniqueAttributes. Why not simply use 
> > entiy guid as the key?

Since objectId is the top-level entity, GUID will not be set in most cases. 
Hence we cannot use GUID.


> On Aug. 14, 2018, 6:34 a.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/impexp/ReplicationDetails.java
> > Lines 37 (patched)
> > 
> >
> > Why is it necessary to store operation & lastUpdate values? Only 
> > lastModifiedTimestamp should be enough, right? I suggest to avoid 
> > ReplicationDetails class completely and instead have the caller:
> > - retrieve AtlasCluster
> > - read additionalInfo.get("REPL_STATUS").get(entityGuid) <== this 
> > should return the modifiedTime of the entity during previous repl-import
> > 
> > In addition, remove AdminResource.getReplicationDetails() and 
> > AtlasBaseClient.getReplicationDetails()

Operation is necessary, we need to know the purpose of the cluster. Given 2 
clusters, one is current cluster and one is data being exported to, there is 
nowhere we denote what is the operation being performed on that cluster.

In scenario, where multiple exports and imports are being performed, this will 
be even more important.


> On Aug. 14, 2018, 6:34 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/impexp/ExportImportAuditService.java
> > Lines 83 (patched)
> > 
> >
> > "result.getEntities() == null" - perhaps you meant "result == null"?

Result is not null always. If there are no entities, then getEntities returns 
null.


- Ashutosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68270/#review207210
---


On Aug. 14, 2018, 6 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68270/
> ---
> 
> (Updated Aug. 14, 2018, 6 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2814
> https://issues.apache.org/jira/browse/ATLAS-2814
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> - New model _ReplicationDetails_ store replication timestamp.
> - _AuditWriter_ updates appropriate _AtlasCluster_ entity with 
> _ReplicationDetails_.
> 
> **REST Call**
> Endpoint: /admin/expimp/audit
> CURL:
> curl -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache" 
> http://localhost:21000/api/atlas/admin/expimp/audit?cluster=cl2
> 
> 
> Diffs
> -
> 
>   webapp/src/test/resources/stocks-base.zip 
> 40c7f37eefb46a944921f6a74a916191704cb9a3 
> 
> 
> Diff: https://reviews.apache.org/r/68270/diff/6/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> New tests added.
> Unit tests related to audits now pause for 5 secs before performing asserts. 
> This should give time for indexes to be created.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 68270: Cluster Entity Updated after Successful Import if Replication Options are Available in Import Request

[Ashutosh Mestry]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68270/
---

(Updated Aug. 14, 2018, 6 p.m.)


Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.


Changes
---

Updates include:
- Changes to expimp Audit REST end point. It now accepts only 1 cluster 
parameter.
- Modified logic for fetching ExportImportAuditEntry search results.
- Addressed review comments.


Bugs: ATLAS-2814
https://issues.apache.org/jira/browse/ATLAS-2814


Repository: atlas


Description (updated)
---

**Approach**
- New model _ReplicationDetails_ store replication timestamp.
- _AuditWriter_ updates appropriate _AtlasCluster_ entity with 
_ReplicationDetails_.

**REST Call**
Endpoint: /admin/expimp/audit
CURL:
curl -X GET -u admin:admin -H "Content-Type: application/json" -H 
"Cache-Control: no-cache" 
http://localhost:21000/api/atlas/admin/expimp/audit?cluster=cl2


Diffs (updated)
-

  webapp/src/test/resources/stocks-base.zip 
40c7f37eefb46a944921f6a74a916191704cb9a3 


Diff: https://reviews.apache.org/r/68270/diff/6/

Changes: https://reviews.apache.org/r/68270/diff/5-6/


Testing
---

**Unit tests**
New tests added.
Unit tests related to audits now pause for 5 secs before performing asserts. 
This should give time for indexes to be created.


Thanks,

Ashutosh Mestry

Review Request 68344: ATLAS-2820 : UI : Add replication audits tab for AtlasCluster entity to show Export & Import audit

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68344/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-2820
https://issues.apache.org/jira/browse/ATLAS-2820


Repository: atlas


Description
---

Add Replication Audits tab in detail page if entity typename is "AtlasCluster".

inside tab show list of audits in table structure using this new API.

/api/atlas/admin/expimp/audit?sourceClusterName=


Diffs
-

  dashboardv2/public/js/collection/VSearchList.js 28c246f24 
  
dashboardv2/public/js/templates/audit/ReplicationAuditTableLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/templates/common/TableLayout_tmpl.html a1104abba 
  dashboardv2/public/js/templates/common/modal.html e000fa8a8 
  dashboardv2/public/js/templates/detail_page/DetailPageLayoutView_tmpl.html 
a322eddb6 
  dashboardv2/public/js/utils/TableLayout.js a2ba0a659 
  dashboardv2/public/js/utils/UrlLinks.js c6335b916 
  dashboardv2/public/js/views/audit/ReplicationAuditTableLayoutView.js 
PRE-CREATION 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js c5f3e6958 
  dashboardv2/public/js/views/search/SearchResultLayoutView.js b4cb36ecc 


Diff: https://reviews.apache.org/r/68344/diff/1/


Testing
---

Done one round of sanity test


Thanks,

keval bhatt

Build failed in Jenkins: Atlas-1.0-IntegrationTests #770

[Apache Jenkins Server]

See 


--
[...truncated 579.04 KB...]
127.0.0.1 - - [14/Aug/2018:15:48:27 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablebzw7djyakl@primary:1534261699000:default.tabletnijq48t8u@primary:153426170-%3E:INSERT:default.tableq2ye1hchji@primary:153426170
 HTTP/1.1" 404 277 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:32 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablebzw7djyakl@primary:1534261699000:default.tabletnijq48t8u@primary:153426170-%3E:INSERT:default.tableq2ye1hchji@primary:153426170
 HTTP/1.1" 200 3025 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:32 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablebzw7djyakl@primary:1534261699000:default.tabletnijq48t8u@primary:153426170-%3E:INSERT:default.tableq2ye1hchji@primary:153426170
 HTTP/1.1" 200 3025 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:32 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/67c9bf7b-a0ab-4cf5-91bc-b2209c313c51 
HTTP/1.1" 200 3025 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:32 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/0fb0747b-3b56-4ded-b776-588eb193ca60 
HTTP/1.1" 200 5562 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:32 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/821308c4-347f-47ab-abec-52892e31f27f 
HTTP/1.1" 200 6348 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:32 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/6fcacb8f-6c58-4845-94ea-49659b9aa126 
HTTP/1.1" 200 6351 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:35 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablebzw7djyakl@primary:1534261699000:default.tabletnijq48t8u@primary:153426170-%3E:INSERT:default.tableq2ye1hchji@primary:153426170
 HTTP/1.1" 200 3025 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:35 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_process?attr:qualifiedName=QUERY:default.tablebzw7djyakl@primary:1534261699000:default.tabletnijq48t8u@primary:153426170-%3E:INSERT:default.tableq2ye1hchji@primary:153426170
 HTTP/1.1" 200 3025 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:35 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/67c9bf7b-a0ab-4cf5-91bc-b2209c313c51 
HTTP/1.1" 200 3025 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:35 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/0fb0747b-3b56-4ded-b776-588eb193ca60 
HTTP/1.1" 200 5562 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:35 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/821308c4-347f-47ab-abec-52892e31f27f 
HTTP/1.1" 200 6348 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:35 +] "GET 
//localhost:31000/api/atlas/v2/entity/guid/6fcacb8f-6c58-4845-94ea-49659b9aa126 
HTTP/1.1" 200 6351 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:38 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tableauitc4peaf@primary
 HTTP/1.1" 200 5180 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:38 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tableauitc4peaf@primary
 HTTP/1.1" 200 5180 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:38 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=db7mmwrl1ene.table1vuis84hnd@primary
 HTTP/1.1" 404 159 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:43 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=db7mmwrl1ene.table1vuis84hnd@primary
 HTTP/1.1" 200 5623 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:43 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=db7mmwrl1ene.table1vuis84hnd@primary
 HTTP/1.1" 200 5623 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:43 +] "GET 
//localhost:31000/api/atlas/v2/lineage/1125bd8b-db56-46eb-b12f-e14883cdcc93?depth=0=INPUT
 HTTP/1.1" 200 1619 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:44 +] "GET 
//localhost:31000/api/atlas/v2/lineage/39079212-c0ac-465a-af10-a868e12ef8f5?depth=0=OUTPUT
 HTTP/1.1" 200 1620 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:46 +] "GET 
//localhost:31000/api/atlas/v2/entity/uniqueAttribute/type/hive_table?attr:qualifiedName=default.tablegzyntnmasr@primary
 HTTP/1.1" 404 154 "-" "Java/1.8.0_172"
127.0.0.1 - - [14/Aug/2018:15:48:51 +] "GET 

[jira] [Updated] (ATLAS-2820) UI : Add replication audits tab for AtlasCluster entity to show Export & Import audit

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-2820:
---
Attachment: ATLAS-2820.patch

> UI : Add replication audits tab for AtlasCluster entity to show Export & 
> Import audit
> -
>
> Key: ATLAS-2820
> URL: https://issues.apache.org/jira/browse/ATLAS-2820
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8.3
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 0.8.3
>
> Attachments: ATLAS-2820-WIP.patch, ATLAS-2820.patch
>
>
> Add Replication Audits tab in detail page if entity typename is 
> "*AtlasCluster*".
> inside tab show list of audits in table structure using this new API.
> /api/atlas/admin/expimp/audit?sourceClusterName=



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : Atlas-master-NoTests #2157

[Apache Jenkins Server]

See 

[jira] [Updated] (ATLAS-2820) UI : Add replication audits tab for AtlasCluster entity to show Export & Import audit

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-2820:
---
Attachment: ATLAS-2820-WIP.patch

> UI : Add replication audits tab for AtlasCluster entity to show Export & 
> Import audit
> -
>
> Key: ATLAS-2820
> URL: https://issues.apache.org/jira/browse/ATLAS-2820
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8.3
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 0.8.3
>
> Attachments: ATLAS-2820-WIP.patch
>
>
> Add Replication Audits tab in detail page if entity typename is 
> "*AtlasCluster*".
> inside tab show list of audits in table structure using this new API.
> /api/atlas/admin/expimp/audit?sourceClusterName=



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2821) Update old atlas-simple-authz-policy.json file with relationship permission attributes

[Nixon Rodrigues (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-2821:
---
Attachment: ATLAS-2821.patch

> Update old atlas-simple-authz-policy.json file with relationship permission 
> attributes
> --
>
> Key: ATLAS-2821
> URL: https://issues.apache.org/jira/browse/ATLAS-2821
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 1.0.0
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: ATLAS-2821.patch
>
>
> This utility will update old atlas-simple-authz-policy.json policy file with 
> relationship default permission attributes for ROLE_ADMIN and DATA_STEWARD.
>  
> Usage : ./atlas_update_simple_auth_json.py 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2821) Update old atlas-simple-authz-policy.json file with relationship permission attributes

[Nixon Rodrigues (JIRA)]

Nixon Rodrigues created ATLAS-2821:
--

 Summary: Update old atlas-simple-authz-policy.json file with 
relationship permission attributes
 Key: ATLAS-2821
 URL: https://issues.apache.org/jira/browse/ATLAS-2821
 Project: Atlas
  Issue Type: Bug
  Components: atlas-webui
Affects Versions: 1.0.0
Reporter: Nixon Rodrigues
Assignee: Nixon Rodrigues
 Fix For: 1.1.0


This utility will update old atlas-simple-authz-policy.json policy file with 
relationship default permission attributes for ROLE_ADMIN and DATA_STEWARD.

 

Usage : ./atlas_update_simple_auth_json.py 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2820) UI : Add replication audits tab for AtlasCluster entity to show Export & Import audit

[Keval Bhatt (JIRA)]

Keval Bhatt created ATLAS-2820:
--

 Summary: UI : Add replication audits tab for AtlasCluster entity 
to show Export & Import audit
 Key: ATLAS-2820
 URL: https://issues.apache.org/jira/browse/ATLAS-2820
 Project: Atlas
  Issue Type: Improvement
Affects Versions: 0.8.3
Reporter: Keval Bhatt
Assignee: Keval Bhatt
 Fix For: 0.8.3


Add Replication Audits tab in detail page if entity typename is 
"*AtlasCluster*".

inside tab show list of audits in table structure using this new API.

/api/atlas/admin/expimp/audit?sourceClusterName=



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: Atlas-1.0-AllTests #763

[Apache Jenkins Server]

See 

--
[...truncated 99.11 KB...]
[INFO] Executing: /bin/sh -c cd 
' && 'git' 
'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: 

[INFO] Storing buildNumber: 705ce81ea2de7b7d2923b8d220575b20efca6bcf at 
timestamp: 1534250520668
[WARNING] Cannot get the branch information from the git repository: 
Detecting the current branch failed: fatal: ref HEAD is not a symbolic ref

[INFO] Executing: /bin/sh -c cd 
' && 'git' 
'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- apache-rat-plugin:0.12:check (rat-check) @ atlas-notification ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 61 implicit excludes (use -debug for more details).
[INFO] Exclude: **/antlr4/**
[INFO] Exclude: **/dependency-reduced-pom.xml
[INFO] Exclude: **/javax.script.ScriptEngineFactory
[INFO] Exclude: .reviewboardrc
[INFO] Exclude: 3party-licenses/**
[INFO] Exclude: **/.cache
[INFO] Exclude: **/.cache-main
[INFO] Exclude: **/.cache-tests
[INFO] Exclude: **/.checkstyle
[INFO] Exclude: **/*.txt
[INFO] Exclude: **/*.json
[INFO] Exclude: .pc/**
[INFO] Exclude: debian/**
[INFO] Exclude: .svn/**
[INFO] Exclude: .git/**
[INFO] Exclude: .gitignore
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.twiki
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.log
[INFO] Exclude: **/target/**
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 36 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 36 licenses.
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
atlas-notification ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-notification ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
atlas-notification ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 21 source files to 

[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-notification ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-notification ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 14 source files to 

[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 

Build failed in Jenkins: Atlas-master-NoTests #2156

[Apache Jenkins Server]

See 

--
[...truncated 410.87 KB...]
[INFO] Exclude: **/.checkstyle
[INFO] Exclude: **/*.txt
[INFO] Exclude: **/*.json
[INFO] Exclude: .pc/**
[INFO] Exclude: debian/**
[INFO] Exclude: .svn/**
[INFO] Exclude: .git/**
[INFO] Exclude: .gitignore
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.twiki
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.log
[INFO] Exclude: **/target/**
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 33 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 33 licenses.
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ atlas-distro 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-distro ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 10 resources to bin
[INFO] Copying 16 resources to conf
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ atlas-distro 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-distro ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-distro ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-distro ---
[INFO] Tests are skipped.
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ atlas-distro ---
[INFO] skipping execute as per configuraion
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ atlas-distro ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.7:attach-descriptor (attach-descriptor) @ 
atlas-distro ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO] 
[INFO] --- maven-assembly-plugin:2.4.1:single (default) @ atlas-distro ---
[INFO] Reading assembly descriptor: 
src/main/assemblies/atlas-hbase-hook-package.xml
[INFO] Reading assembly descriptor: 
src/main/assemblies/atlas-hive-hook-package.xml
[INFO] Reading assembly descriptor: 
src/main/assemblies/atlas-falcon-hook-package.xml
[INFO] Reading assembly descriptor: 
src/main/assemblies/atlas-sqoop-hook-package.xml
[INFO] Reading assembly descriptor: 
src/main/assemblies/atlas-storm-hook-package.xml
[INFO] Reading assembly descriptor: 
src/main/assemblies/atlas-kafka-hook-package.xml
[INFO] Reading assembly descriptor: src/main/assemblies/atlas-server-package.xml
[INFO] Reading assembly descriptor: src/main/assemblies/standalone-package.xml
[INFO] Reading assembly descriptor: src/main/assemblies/src-package.xml
[INFO] Reading assembly descriptor: src/main/assemblies/migration-exporter.xml
[INFO] Building tar: 

[INFO] Copying files to 

[WARNING] Assembly file: 

 is not a regular file (it may be a directory). It cannot be attached to the 
project build for installation or deployment.
[INFO] Building tar: 

[INFO] Copying files to 

[WARNING] Assembly file: 

Re: Review Request 68270: Cluster Entity Updated after Successful Import if Replication Options are Available in Import Request

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68270/#review207210
---




intg/src/main/java/org/apache/atlas/model/clusterinfo/AtlasCluster.java
Lines 80 (patched)


line #80 is not necessary, as this will be handled in #89 below.



intg/src/main/java/org/apache/atlas/model/clusterinfo/AtlasCluster.java
Lines 85 (patched)


Use of stringified-AtlasObjectId as key doesn't sound good - since 
AtlasObjectId has guid/type/state/uniqueAttributes. Why not simply use entiy 
guid as the key?



intg/src/main/java/org/apache/atlas/model/impexp/ReplicationDetails.java
Lines 37 (patched)


Why is it necessary to store operation & lastUpdate values? Only 
lastModifiedTimestamp should be enough, right? I suggest to avoid 
ReplicationDetails class completely and instead have the caller:
- retrieve AtlasCluster
- read additionalInfo.get("REPL_STATUS").get(entityGuid) <== this should 
return the modifiedTime of the entity during previous repl-import

In addition, remove AdminResource.getReplicationDetails() and 
AtlasBaseClient.getReplicationDetails()



repository/src/main/java/org/apache/atlas/repository/impexp/ExportImportAuditService.java
Lines 83 (patched)


"result.getEntities() == null" - perhaps you meant "result == null"?



repository/src/main/java/org/apache/atlas/repository/impexp/ExportImportAuditService.java
Lines 88 (patched)


Instead of this 'for' loop to retrieve each entity from db, it will be 
efficient to covert entityHeader (AtlasEntityHeader) into 
ExportImportAuditEntry object.

Search API can return necessary attributes (to populate 
ExportImportAuditEntry); just update searchParameters in #77 to include list of 
attributes: searchParameters.setAttribute(attributeNames).



repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
Lines 115 (patched)


Why pay the cost of AtlasType.toJson() call, when debug is not enabled? 
Replace lines #115 - #121 with the following:

  if (LOG.isDebugEnabled()) {
LOG.debug("   => transforms: {}", AtlasType.toJson(importTransform));
  }

Same for such usage in ExportService.java as well.


- Madhan Neethiraj


On Aug. 13, 2018, 7:07 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68270/
> ---
> 
> (Updated Aug. 13, 2018, 7:07 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2814
> https://issues.apache.org/jira/browse/ATLAS-2814
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> - New model _ReplicationDetails_ store replication timestamp.
> - _AuditWriter_ updates appropriate _AtlasCluster_ entity with 
> _ReplicationDetails_.
> 
> **REST Call**
> Endpoint: /admin/expimp/audit
> CURL:
> curl -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache" 
> http://localhost:21000/api/atlas/admin/expimp/audit?sourceClusterName=cl2
> 
> 
> Diffs
> -
> 
>   client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java 
> f73ba2c6e6c9d05e86eb6c17f923a6dda1df5dd9 
>   intg/src/main/java/org/apache/atlas/model/clusterinfo/AtlasCluster.java 
> 3ce50e38b3af45521c2c83d0e6c05398747bcf86 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasExportResult.java 
> 85a606c7d3b2158238b2a6defbb9185883434d78 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportResult.java 
> bfb7637711e08e17fd5f5b16bf121ac541b631a4 
>   intg/src/main/java/org/apache/atlas/model/impexp/ReplicationDetails.java 
> PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/AuditsWriter.java 
> 6a3fbecda84dc48e0df1359c1b5a248958db0a4c 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ClusterService.java
>  fd8e2bfe6bbdb7ae5e66ab67ecdf94e9527d9b09 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportImportAuditService.java
>  e90b6b942d359f78ccf59ee955c4667f2033e6c4 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
>  b15f828e470eaee972a4ee3f8447f404f813c699 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
>  98ef389bd1bf8f87d23a06dc231d4d93f06b2231 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasClusterDTO.java 
>