Re: Review Request 60863: ATLAS-1952 : UI : Render Search table using referredEntity.

2017-07-14 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60863/#review180604
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 14, 2017, 12:34 p.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60863/
> ---
> 
> (Updated July 14, 2017, 12:34 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1952
> https://issues.apache.org/jira/browse/ATLAS-1952
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> UI becomes more chatty when user select dynamic columns example 
> Column,Table,db so ATLAS-1947(from server side) will fix this issue by adding 
> referredEntity attribute in search result.
> 
> UI will use this referredEntity and render the table.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/collection/VSearchList.js 6291838 
>   dashboardv2/public/js/utils/CommonViewFunction.js a04fb3f 
>   dashboardv2/public/js/utils/Utils.js 9071df3 
>   dashboardv2/public/js/views/entity/EntityDetailTableLayoutView.js 87c5dcd 
>   dashboardv2/public/js/views/search/SearchResultLayoutView.js dba27fa 
> 
> 
> Diff: https://reviews.apache.org/r/60863/diff/1/
> 
> 
> Testing
> ---
> 
> Tested and checked in network no extra call is made while selecting dynamic 
> columns.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Build failed in Jenkins: apache-atlas-nightly #806

2017-07-14 Thread Apache Jenkins Server 
See 


Changes:

[madhan] ATLAS-1940: fix to remove duplicate type which causes Atlas server to

[madhan] ATLAS-1908: updated to use existing Kafka consumer properties when

--
[...truncated 543.70 KB...]
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "POST 
/api/atlas/entities/2dcae9d3-2a2e-43b9-ad03-902f6f08f124/traits HTTP/1.1" 201 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "GET 
/api/atlas/entities/2dcae9d3-2a2e-43b9-ad03-902f6f08f124/traitDefinitions/PII_TraitX7vnSYCHUH
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "GET 
/api/atlas/entities/2dcae9d3-2a2e-43b9-ad03-902f6f08f124/traitDefinitions 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "GET 
/api/atlas/entities/4965fc49-ba89-49e8-8cdd-54bdfc3f17aa/traits HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:58 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST 
/api/atlas/entities/41c75895-92a2-4a69-b39c-0478094fb327 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "GET 
/api/atlas/entities/41c75895-92a2-4a69-b39c-0478094fb327 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST 
/api/atlas/entities/qualifiedName?type=hive_table=qualifiedName=tableuKGfM1wzRG
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "GET 
/api/atlas/entities/41c75895-92a2-4a69-b39c-0478094fb327 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "GET 
/api/atlas/entities/65a96be2-ecbf-466c-808f-9b9beba25f65/audit?count=10 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
400 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "GET 
/api/atlas/v2/types/typedef/name/DA1YlHRB4A HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "GET 
/api/atlas/entities/b8e80a60-814f-4ebb-bf04-f155138d9c4b HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "GET 
/api/atlas/v2/entity/guid/a0ca2872-7daf-4afb-afb4-5a569f43118d HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST 
/api/atlas/v2/entity/guid/random/classifications HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/v2/entity/bulk/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:15:59 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00 +] "DELETE 
/api/atlas/v2/entity/bulk/?guid=64a46400-94dd-4a71-aab1-320227221607=8e166eb5-5c33-4a12-94bf-bd5e92b5f0c8
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00 +] "DELETE 
/api/atlas/v2/entity/uniqueAttribute/type/hive_db_v2?attr:name=dbFMfrHMpy1y%E0%A7%8F%EE%9C%83%EC%86%B0%E5%9F%A2%E5%B7%99%ED%8C%92%E6%B5%AF%EC%BE%BC%F0%9B%86%81
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00 +] "DELETE 
/api/atlas/v2/entity/guid/random/classification/blah_trait HTTP/1.1" 400 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db_v2+where+name%3D'dbFMfrHMpy1y'=10=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [14/Jul/2017:23:16:00

Re: Review Request 60852: ATLAS-1947: Improvements for ATLAS-1880 and some change to AtlasSearchResult

2017-07-14 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60852/#review180602
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 14, 2017, 10:49 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60852/
> ---
> 
> (Updated July 14, 2017, 10:49 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1947
> https://issues.apache.org/jira/browse/ATLAS-1947
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> The search result will now include the following
> 
> 1. guid -> referredEntity
> 2. Any valid attribute found in the search filter will be included for each 
> entity in the result
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java 
> 9513dcb1 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> 30855dcf 
>   
> repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java
>  030a9577 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  5068fa5f 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java 
> 1056b3ef 
>   repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/SearchPipeline.java 
> 0f91b2d1 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/SolrStep.java 6a5dd5ae 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  4ac00a32 
>   repository/src/main/java/org/apache/atlas/util/SearchTracker.java 15a8c203 
>   repository/src/test/java/org/apache/atlas/TestModules.java d0da0303 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 1a9f57a3 
>   webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java dde300e2 
> 
> 
> Diff: https://reviews.apache.org/r/60852/diff/3/
> 
> 
> Testing
> ---
> 
> mvn clean package runs successfully
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 60852: ATLAS-1947: Improvements for ATLAS-1880 and some change to AtlasSearchResult

2017-07-14 Thread Apoorv Naik 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60852/
---

(Updated July 14, 2017, 10:49 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Summary (updated)
-

ATLAS-1947: Improvements for ATLAS-1880 and some change to AtlasSearchResult


Bugs: ATLAS-1947
https://issues.apache.org/jira/browse/ATLAS-1947


Repository: atlas


Description
---

The search result will now include the following

1. guid -> referredEntity
2. Any valid attribute found in the search filter will be included for each 
entity in the result


Diffs
-

  intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java 
9513dcb1 
  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
30855dcf 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
030a9577 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
5068fa5f 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java 1056b3ef 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/SearchPipeline.java 
0f91b2d1 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/SolrStep.java 6a5dd5ae 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 4ac00a32 
  repository/src/main/java/org/apache/atlas/util/SearchTracker.java 15a8c203 
  repository/src/test/java/org/apache/atlas/TestModules.java d0da0303 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
1a9f57a3 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java dde300e2 


Diff: https://reviews.apache.org/r/60852/diff/3/


Testing
---

mvn clean package runs successfully


Thanks,

Apoorv Naik

Re: Review Request 60852: ATLAS-1947: Search results improvment

2017-07-14 Thread Apoorv Naik 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60852/
---

(Updated July 14, 2017, 10:48 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Changes
---

Search code structure has changed from SolrStep and GremlinStep => 
FullTextProcessor, EntityProcessor, ClassificationProcessor

These can be chained one after other and internally each determine whether to 
execute indexQuery or regular graph query.
PipelineContext => SearchContext : Is responsible for determining the order of 
the processors as well


Bugs: ATLAS-1947
https://issues.apache.org/jira/browse/ATLAS-1947


Repository: atlas


Description
---

The search result will now include the following

1. guid -> referredEntity
2. Any valid attribute found in the search filter will be included for each 
entity in the result


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java 
9513dcb1 
  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
30855dcf 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
030a9577 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
5068fa5f 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java 1056b3ef 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/SearchPipeline.java 
0f91b2d1 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/SolrStep.java 6a5dd5ae 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 4ac00a32 
  repository/src/main/java/org/apache/atlas/util/SearchTracker.java 15a8c203 
  repository/src/test/java/org/apache/atlas/TestModules.java d0da0303 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
1a9f57a3 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java dde300e2 


Diff: https://reviews.apache.org/r/60852/diff/3/

Changes: https://reviews.apache.org/r/60852/diff/2-3/


Testing
---

mvn clean package runs successfully


Thanks,

Apoorv Naik

[jira] [Commented] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-14 Thread Sharmadha Sainath (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087925#comment-16087925
 ] 

Sharmadha Sainath commented on ATLAS-1939:
--

Attached a patch that would not throw NPE when JSON is not provided for request 
parameter . 

The following request with the patch now would succeed :
{code}
curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F data=@t5.zip 
"http://localhost:21000/api/atlas/admin/import;
{code}

In the patch , added an empty JSON to request object if request is null or 
empty.

CC : [~ashutoshm] [~mad...@apache.org] [~ayubkhan]

> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: ATLAS-1939.patch, Import_error_NPE2.txt, Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-14 Thread Ashutosh Mestry (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087923#comment-16087923
 ] 

Ashutosh Mestry commented on ATLAS-1939:


I think its a good fix!

> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: ATLAS-1939.patch, Import_error_NPE2.txt, Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-14 Thread Sharmadha Sainath (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sharmadha Sainath updated ATLAS-1939:
-
Attachment: ATLAS-1939.patch

> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: ATLAS-1939.patch, Import_error_NPE2.txt, Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60857: ATLAS-1950: Import API: Improvement: Specify Supertypes in Import Transforms

2017-07-14 Thread Ashutosh Mestry 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60857/
---

(Updated July 14, 2017, 6 p.m.)


Review request for atlas, Ayub Pathan and Madhan Neethiraj.


Changes
---

Updates include:
- Updated request with issue id.


Summary (updated)
-

ATLAS-1950: Import API: Improvement: Specify Supertypes in Import Transforms


Bugs: ATLAS-1950
https://issues.apache.org/jira/browse/ATLAS-1950


Repository: atlas


Description
---

**Background**
Existing import transforms expects that the user specify each type and 
attribute name that needs to be transformed. This may cause large number of 
transform enumerating individual types to be specified. Specifying individual 
transforms may be tedious and perhaps error prone.

**Solution**
This feature improves on that case by allowing user to specify super types. 
That way the sub types of that type will automatically get the transform(s) 
applied.

**Implementation**
User can now specify transforms like:

```javascript
{
"options": {
"transforms": "{ \"Asset\": { \"qualifiedName\": [ 
\"replace:@cl1:@cl2\" ] } }"
}
}
```

At runtime, the subtypes of this type are enumerated. The attribute and 
tranform map of the super type are now associated with all the sub-types.


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 
92217178 
  
repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransforms.java
 63e53c50 
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
 de8e7ef3 


Diff: https://reviews.apache.org/r/60857/diff/1/


Testing
---

**Unit tests**
Unit test verifying this behavior were added.

**Functional tests**
Regular imports were run.


Thanks,

Ashutosh Mestry

[jira] [Updated] (ATLAS-1950) Import Transform option using supertype instead of a specific type

2017-07-14 Thread Ashutosh Mestry (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-1950:
---
Attachment: ATLAS-1950.patch

> Import Transform option using supertype instead of a specific type
> --
>
> Key: ATLAS-1950
> URL: https://issues.apache.org/jira/browse/ATLAS-1950
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
> Attachments: ATLAS-1950.patch
>
>
> Users can provide a tranforms option while import to replace @cl1 in 
> hive_table to @cl2 using the following JSON.
> {code}
> {
> "options": {
> "transforms": "{ \"hive_table\": { \"qualifiedName\": [ 
> \"replace:@cl1:@cl2\" ] } }"
> }
> }
> {code}
> It would be easy to specify a super type like 'Asset' to transform all types 
> in the export items which inherit from the super type to have "@cl1" replaced 
> with "@cl2" like
> {code}
> {
> "options": {
> "transforms": "{ \"Asset\": { \"qualifiedName\": [ 
> \"replace:@cl1:@cl2\" ] } }"
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60868: ATLAS-1940 :- Fix to remove duplicate type which causes Atlas server to fails on startup.

2017-07-14 Thread Ashutosh Mestry 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60868/#review180535
---


Ship it!




Ship It!

- Ashutosh Mestry


On July 14, 2017, 3:13 p.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60868/
> ---
> 
> (Updated July 14, 2017, 3:13 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1940
> https://issues.apache.org/jira/browse/ATLAS-1940
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Problem:- Atlas server fails to initialize types on startup from graph store 
> due to duplicate type registered.
> 
> Fix :- Added removeDuplicateTypeIfAny method which is called init of typeDef 
> Store to remove duplicate types if any.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  9d9c59d 
> 
> 
> Diff: https://reviews.apache.org/r/60868/diff/1/
> 
> 
> Testing
> ---
> 
> Exiting UT and IT are running fine.
> Quick Start script running fine.
> Created new tags & types. 
> Also tried creating duplicate type from rest client and UI which failed to 
> create duplicates.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

[jira] [Updated] (ATLAS-1940) Atlas server fails to initialize types on startup from graph store due to duplicate type registered.

2017-07-14 Thread Nixon Rodrigues (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-1940:
---
Attachment: ATLAS-1940.patch

> Atlas server fails to initialize types on startup from graph store due to 
> duplicate type registered.
> 
>
> Key: ATLAS-1940
> URL: https://issues.apache.org/jira/browse/ATLAS-1940
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 0.8-incubating
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Blocker
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1940.patch
>
>
> Types are not getting restored from typesRegistry during Atlas startup due 
> duplicate type issue.
> Below is exception trace.
> {code}
> 2017-07-07 13:35:23,088 INFO  - [main:] ~ Restoring type CLASS.Process.null 
> (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,099 INFO  - [main:] ~ Restoring type 
> CLASS.falcon_process.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,122 INFO  - [main:] ~ Restoring type TRAIT.Metric.Metric 
> Trait (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,132 INFO  - [main:] ~ Restoring type 
> CLASS.Referenceable.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,140 INFO  - [main:] ~ Restoring type 
> CLASS.hive_storagedesc.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,203 INFO  - [main:] ~ Restoring type 
> CLASS.hive_table.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,276 INFO  - [main:] ~ Restoring type 
> CLASS.falcon_feed_creation.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,288 INFO  - [main:] ~ Restoring type 
> CLASS.sqoop_dbdatastore.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,399 INFO  - [main:] ~ Number of types got from transient 
> type system: 49 (DefaultMetadataService:168)
> 2017-07-07 13:35:23,406 INFO  - [main:] ~ Restored type system from the store 
> (DefaultMetadataService:161)
> 2017-07-07 13:35:23,407 INFO  - [main:] ~ Global indexes already exist for 
> graph (GraphBackedSearchIndexer:141)
> 2017-07-07 13:35:23,753 WARN  - [main:] ~ Invalid constraint ownedRef for 
> attribute hive_storagedesc.serdeInfo (AtlasTypeDefGraphStore:689)
> 2017-07-07 13:35:23,754 WARN  - [main:] ~ Invalid constraint ownedRef for 
> attribute hive_table.aliases (AtlasTypeDefGraphStore:689)
> 2017-07-07 13:35:23,759 ERROR - [main:] ~ failed to initialize types from 
> graph store (AtlasTypeDefGraphStoreV1:77)
> org.apache.atlas.exception.AtlasBaseException: Given type TaxonomyTerm 
> already exists
>   at 
> org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypeWithNoRefResolve(AtlasTypeRegistry.java:573)
>   at 
> org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypesWithNoRefResolve(AtlasTypeRegistry.java:611)
>   at 
> org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypes(AtlasTypeRegistry.java:407)
>   at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.init(AtlasTypeDefGraphStore.java:105)
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.init(AtlasTypeDefGraphStoreV1.java:107)
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.(AtlasTypeDefGraphStoreV1.java:75)
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be.()
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be$$FastClassByGuice$$a11fb32d.newInstance()
>   at 
> com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265)
>   at 
> com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)
>   at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
>   at 
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
>   at 
> com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)
>   at 
> com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
>   at 
> com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205)
>   at 
>

[jira] [Commented] (ATLAS-1940) Atlas server fails to initialize types on startup from graph store due to duplicate type registered.

2017-07-14 Thread Nixon Rodrigues (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087453#comment-16087453
 ] 

Nixon Rodrigues commented on ATLAS-1940:


[~ayubkhan], Not sure whether this issue similiar to ATLAS-1472, but fix for 
this issue is to remove the duplicate types in this patch.

> Atlas server fails to initialize types on startup from graph store due to 
> duplicate type registered.
> 
>
> Key: ATLAS-1940
> URL: https://issues.apache.org/jira/browse/ATLAS-1940
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 0.8-incubating
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Blocker
> Fix For: 0.9-incubating, 0.8.1-incubating
>
>
> Types are not getting restored from typesRegistry during Atlas startup due 
> duplicate type issue.
> Below is exception trace.
> {code}
> 2017-07-07 13:35:23,088 INFO  - [main:] ~ Restoring type CLASS.Process.null 
> (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,099 INFO  - [main:] ~ Restoring type 
> CLASS.falcon_process.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,122 INFO  - [main:] ~ Restoring type TRAIT.Metric.Metric 
> Trait (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,132 INFO  - [main:] ~ Restoring type 
> CLASS.Referenceable.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,140 INFO  - [main:] ~ Restoring type 
> CLASS.hive_storagedesc.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,203 INFO  - [main:] ~ Restoring type 
> CLASS.hive_table.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,276 INFO  - [main:] ~ Restoring type 
> CLASS.falcon_feed_creation.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,288 INFO  - [main:] ~ Restoring type 
> CLASS.sqoop_dbdatastore.null (GraphBackedTypeStore:267)
> 2017-07-07 13:35:23,399 INFO  - [main:] ~ Number of types got from transient 
> type system: 49 (DefaultMetadataService:168)
> 2017-07-07 13:35:23,406 INFO  - [main:] ~ Restored type system from the store 
> (DefaultMetadataService:161)
> 2017-07-07 13:35:23,407 INFO  - [main:] ~ Global indexes already exist for 
> graph (GraphBackedSearchIndexer:141)
> 2017-07-07 13:35:23,753 WARN  - [main:] ~ Invalid constraint ownedRef for 
> attribute hive_storagedesc.serdeInfo (AtlasTypeDefGraphStore:689)
> 2017-07-07 13:35:23,754 WARN  - [main:] ~ Invalid constraint ownedRef for 
> attribute hive_table.aliases (AtlasTypeDefGraphStore:689)
> 2017-07-07 13:35:23,759 ERROR - [main:] ~ failed to initialize types from 
> graph store (AtlasTypeDefGraphStoreV1:77)
> org.apache.atlas.exception.AtlasBaseException: Given type TaxonomyTerm 
> already exists
>   at 
> org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypeWithNoRefResolve(AtlasTypeRegistry.java:573)
>   at 
> org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypesWithNoRefResolve(AtlasTypeRegistry.java:611)
>   at 
> org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypes(AtlasTypeRegistry.java:407)
>   at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.init(AtlasTypeDefGraphStore.java:105)
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.init(AtlasTypeDefGraphStoreV1.java:107)
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.(AtlasTypeDefGraphStoreV1.java:75)
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be.()
>   at 
> org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be$$FastClassByGuice$$a11fb32d.newInstance()
>   at 
> com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265)
>   at 
> com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111)
>   at 
> com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)
>   at 
> com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)
>   at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
>   at 
> com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
>   at 
> com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
>   at 
> com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)
>   at 
> com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
>   at 
>

Review Request 60868: ATLAS-1940 :- Fix to remove duplicate type which causes Atlas server to fails on startup.

2017-07-14 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60868/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-1940
https://issues.apache.org/jira/browse/ATLAS-1940


Repository: atlas


Description
---

Problem:- Atlas server fails to initialize types on startup from graph store 
due to duplicate type registered.

Fix :- Added removeDuplicateTypeIfAny method which is called init of typeDef 
Store to remove duplicate types if any.


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
 9d9c59d 


Diff: https://reviews.apache.org/r/60868/diff/1/


Testing
---

Exiting UT and IT are running fine.
Quick Start script running fine.
Created new tags & types. 
Also tried creating duplicate type from rest client and UI which failed to 
create duplicates.


Thanks,

Nixon Rodrigues

[jira] [Created] (ATLAS-1953) Introduce the ability for icons to be associated with EntityDefs and RelaitonshipDefs

2017-07-14 Thread David Radley (JIRA) 
David Radley created ATLAS-1953:
---

 Summary: Introduce the ability for icons to be associated with 
EntityDefs and RelaitonshipDefs
 Key: ATLAS-1953
 URL: https://issues.apache.org/jira/browse/ATLAS-1953
 Project: Atlas
  Issue Type: Improvement
Reporter: David Radley
Assignee: David Radley


Introduce the ability for icons to be defined in EntityDefs and 
RelaitonshipDefs - these icons would be picked up by instances. This 
facilitates visualization of entities and the like.  The icon would have type 
string and its value would be a url or a path relative to the webserver base 
url. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1952) UI : Render Search table using referredEntity.

2017-07-14 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1952:
---
Description: 
UI becomes more chatty when user select dynamic columns example Column, Table, 
db so ATLAS-1947(from the server side) will fix this issue by adding 
referredEntity attribute in the search result.

UI will use this referredEntity and render the table.

  was:
UI becomes more chatty when user select dynamic columns example Column,Table,db 
so ATLAS-1947(from server side) will fix this issue by adding referredEntity 
attribute in search result.

UI will use this referredEntity and render the table.


> UI : Render Search table using referredEntity.
> --
>
> Key: ATLAS-1952
> URL: https://issues.apache.org/jira/browse/ATLAS-1952
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.9-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1952.patch
>
>
> UI becomes more chatty when user select dynamic columns example Column, 
> Table, db so ATLAS-1947(from the server side) will fix this issue by adding 
> referredEntity attribute in the search result.
> UI will use this referredEntity and render the table.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1952) UI : Render Search table using referredEntity.

2017-07-14 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1952:
---
Description: 
UI becomes more chatty when user select dynamic columns example Column,Table,db 
so ATLAS-1947(from server side) will fix this issue by adding referredEntity 
attribute in search result.

UI will use this referredEntity and render the table.

> UI : Render Search table using referredEntity.
> --
>
> Key: ATLAS-1952
> URL: https://issues.apache.org/jira/browse/ATLAS-1952
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.9-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1952.patch
>
>
> UI becomes more chatty when user select dynamic columns example 
> Column,Table,db so ATLAS-1947(from server side) will fix this issue by adding 
> referredEntity attribute in search result.
> UI will use this referredEntity and render the table.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1952) UI : Render Search table using referredEntity.

2017-07-14 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1952:
---
Attachment: ATLAS-1952.patch

> UI : Render Search table using referredEntity.
> --
>
> Key: ATLAS-1952
> URL: https://issues.apache.org/jira/browse/ATLAS-1952
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.9-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1952.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1952) UI : Render Search table using referredEntity.

2017-07-14 Thread Keval Bhatt (JIRA) 
Keval Bhatt created ATLAS-1952:
--

 Summary: UI : Render Search table using referredEntity.
 Key: ATLAS-1952
 URL: https://issues.apache.org/jira/browse/ATLAS-1952
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.9-incubating
Reporter: Keval Bhatt
Assignee: Keval Bhatt
 Fix For: 0.9-incubating, 0.8.1-incubating






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1951) Regression: Any REST API request without user credentials results in 302 redirect to login.jsp. Actually, the correct response should be 401.

2017-07-14 Thread Ayub Pathan (JIRA) 
Ayub Pathan created ATLAS-1951:
--

 Summary: Regression: Any REST API request without user credentials 
results in 302 redirect to login.jsp. Actually, the correct response should be 
401.
 Key: ATLAS-1951
 URL: https://issues.apache.org/jira/browse/ATLAS-1951
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan
Priority: Critical
 Fix For: 0.9-incubating


Regression: Any REST API request to atlas without user credentials results in 
302 redirect( login.jsp), but the actual response code should be 401 
unauthorized. 

This issue could have been introduced as part of new spring  changes.

For example:
{noformat}
curl -v -X GET "http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000;
* Rebuilt URL to: http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/
*   Trying 172.27.56.2...
* Connected to ctr-e133-1493418528701-181199-01-02.hwx.site (172.27.56.2) 
port 21000 (#0)
> GET / HTTP/1.1
> Host: ctr-e133-1493418528701-181199-01-02.hwx.site:21000
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Fri, 14 Jul 2017 11:16:42 GMT
< Set-Cookie: ATLASSESSIONID=1i0rxnm66dd3h17xyhvstk0vck;Path=/;HttpOnly
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< X-Frame-Options: DENY
< Location: 
http://ctr-e133-1493418528701-181199-01-02.hwx.site:21000/login.jsp
< Content-Length: 0
< Server: Jetty(9.2.12.v20150709)
<
* Connection #0 to host ctr-e133-1493418528701-181199-01-02.hwx.site left 
intact
{noformat}

CC [~skoneru]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1698) Create Glossary OMAS API

2017-07-14 Thread Stefhan van Helvoirt (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087180#comment-16087180
 ] 

Stefhan van Helvoirt commented on ATLAS-1698:
-

HI [~davidrad], hereby my feedback hopefully its helpfull.

Page 1: you mention
- 'Create a new word'
- 'Create a term'

What is the difference between 'word' and 'term'? 

I really like the picture you put on page 2, i see it now in the current work 
we do with creating a glossary. It would be highly desirable if the workflow 
relating the approval process of terms could be adjusted based on the type of 
change that occured. For example, for the creation of a term and the setting of 
a definition you would like to have the Definition Owner involved. However when 
linking terms together to state in which context they are used, it might not be 
needed to follow this same approval process, or at least not need the same 
people involved. 
Might not be relevant for this JIRA, but something to take into consideration 
in the overal design.

Page 3:
What is a visualizable term? Do also non-visualizable terms exsist? 

"Need to ensure we have a way to scope what is displayed", would be good if you 
could group sets of terms in a category/collection or whatever you call it and 
that a graph can be created based on this context. Sometimes you want to have a 
graph from a hierarchical perspective, in other situations a functional view 
might be more desirable. Should be possible to have both. 

Page 4: I don't understand the mentioning of ETL / ELT in this context. Are you 
referring here to using ETL/ELT for creating metadata? 








> Create Glossary OMAS API
> 
>
> Key: ATLAS-1698
> URL: https://issues.apache.org/jira/browse/ATLAS-1698
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: David Radley
>Assignee: David Radley
>  Labels: VirtualDataConnector
> Attachments: apidocs-3.zip, Atlas Glossary OMAS API proposal v1.0 
> .pdf, Atlas Glossary OMAS API proposal v1.1.pdf
>
>
> The Glossary OMAS provides a specialized API for glossary applications to 
> retrieve and store their glossary metadata and link assets of different types 
> to these glossary entries.
> The Glossary OMAS makes heavy use of the Area 2 open metadata model.  See 
> https://cwiki.apache.org/confluence/display/ATLAS/Area+2+-+Glossary



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60705: ATLAS-1927 : UI - changing the look and feel of attribute in details page.

2017-07-14 Thread Kalyani Kashikar 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60705/
---

(Updated July 14, 2017, 10:08 a.m.)


Review request for atlas, keval bhatt, Madhan Neethiraj, and Nixon Rodrigues.


Changes
---

Updated patch contains changes to render the Tags attribute data in table 
format. Data is rendered in sorted order and Date/time format is shown for 
'Date' data-type.


Bugs: ATLAS-1927
https://issues.apache.org/jira/browse/ATLAS-1927


Repository: atlas


Description
---

Show the attributes of entities in tag format instead of showing comma(,) 
separator.


Diffs (updated)
-

  dashboardv2/public/css/scss/tag.scss fc0652b 
  dashboardv2/public/js/router/Router.js 8a1d241 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js c45755f 
  dashboardv2/public/js/views/tag/TagDetailTableLayoutView.js 45cdd6b 
  dashboardv2/public/js/views/tag/addTagModalView.js 97d80bd 


Diff: https://reviews.apache.org/r/60705/diff/2/

Changes: https://reviews.apache.org/r/60705/diff/1-2/


Testing
---

verified the attributes of entities is showing in tag format instead of showing 
comma(,) separator.


Thanks,

Kalyani Kashikar

[jira] [Updated] (ATLAS-1908) Updating old Kafka consumer api properties to reflect change in new KafkaConsumer configs for Atlas.

2017-07-14 Thread Nixon Rodrigues (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-1908:
---
Attachment: ATLAS-1908-1.patch

> Updating old Kafka consumer api properties to reflect change in new 
> KafkaConsumer configs for Atlas.
> 
>
> Key: ATLAS-1908
> URL: https://issues.apache.org/jira/browse/ATLAS-1908
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
> Attachments: ATLAS-1908-1.patch, ATLAS-1908.patch
>
>
> In ATLAS-1766, the old Kafka consumer api was updated with new Kafka consumer 
> api.
> Due to this update, there were changes in configs.
>  
> New property:
> atlas.kafka.session.timeout.ms=3
>  
> Renamed property:
> atlas.kafka.auto.commit.enable=false to atlas.kafka.enable.auto.commit=false
>  
> New default value:
> atlas.kafka.auto.offset.reset=smallest to 
> atlas.kafka.auto.offset.reset=earliest
> Need to handle old configs, so that Kafka consumer starts gracefully even if 
> new configs are not added explicitly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1927) UI - changing the look and feel of attribute in details page.

2017-07-14 Thread Kalyani Kashikar (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kalyani Kashikar updated ATLAS-1927:

Attachment: ATLAS-1927.1.patch

> UI - changing the look and feel of attribute in details page.
> -
>
> Key: ATLAS-1927
> URL: https://issues.apache.org/jira/browse/ATLAS-1927
> Project: Atlas
>  Issue Type: Bug
>Reporter: Kalyani Kashikar
>Assignee: Kalyani Kashikar
> Attachments: ATLAS-1927.1.patch, ATLAS-1927.patch
>
>
> Show the attributes of entities in tag format instead of showing comma(,) 
> separator.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60597: ATLAS-1908 : - Updating old Kafka consumer api properties to reflect change in new KafkaConsumer configs for Atlas

2017-07-14 Thread Nixon Rodrigues 


> On July 13, 2017, 5:23 p.m., Sarath Subramanian wrote:
> > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java
> > Line 126 (original), 127 (patched)
> > 
> >
> > can the new kafka consumer handle both "smallest" and "earliest" value 
> > for auto.offset.reset?

the new kafka consumer can handle only earliest, smallest causes exception 
while initialization of kafkaconsumer. so set earliet as default which was set 
earlier.


> On July 13, 2017, 5:23 p.m., Sarath Subramanian wrote:
> > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java
> > Lines 133 (patched)
> > 
> >
> > should this be subsetConfiguration.getLong("session.timeout.ms", 3000)?

Tried your suggestion but there was a parse error (below). The kafka config is 
expecting the long value in string format

 272 org.apache.kafka.common.config.ConfigException: Invalid value 3 
for configuration session.timeout.ms: Expected value to be an number.


> On July 13, 2017, 5:23 p.m., Sarath Subramanian wrote:
> > notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java
> > Line 102 (original), 102 (patched)
> > 
> >
> > why reduced poll timeout to 100L from 1000?

The return output of from poll method is mocked, so poll timeout will not 
matter as such, so set the timeout it less value to reduce execution time.


- Nixon


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60597/#review180445
---


On July 14, 2017, 9:52 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60597/
> ---
> 
> (Updated July 14, 2017, 9:52 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1908
> https://issues.apache.org/jira/browse/ATLAS-1908
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch handles old kafka consumer configs values to be added as new 
> configs if new configs are not present , so that Kafka consumer starts 
> gracefully even if new configs are not added explicitly.
> 
> eg. atlas.kafka.auto.commit.enable=false to 
> atlas.kafka.enable.auto.commit=false
> atlas.kafka.auto.offset.reset=smallest to 
> atlas.kafka.auto.offset.reset=earliest
> atlas.kafka.session.timeout.ms=3
> 
> Also added poll.timeout.ms in configs so kafka consumer can be 
> configurable.This involve change in AtlasKafkaConsumer receive interface.
> 
> 
> Diffs
> -
> 
>   distro/src/conf/atlas-application.properties c3213df 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> 9c15243 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
> 366c8a7 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  22e40f9 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 
> 70059cb 
>   
> notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 
> c791d43 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  8324b57 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  0dea0e2 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
>  650ca0a 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> c036cfa 
> 
> 
> Diff: https://reviews.apache.org/r/60597/diff/2/
> 
> 
> Testing
> ---
> 
> Executed UT/IT's using mvn clean install.
> Tested KafkaConsumer with Hive Hook by creating table in hive.
> Atlas starts gracefully and Kafka consumer is up without error when new 
> properties are not added.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 60597: ATLAS-1908 : - Updating old Kafka consumer api properties to reflect change in new KafkaConsumer configs for Atlas

2017-07-14 Thread Nixon Rodrigues 


> On July 13, 2017, 5:03 a.m., Ashutosh Mestry wrote:
> > distro/src/conf/atlas-application.properties
> > Lines 82 (patched)
> > 
> >
> > Should this be 1000 ms? My thinking is 100 ms will be too short.

Yes 100 ms is small, i have changed it 1000 ms, some tuning might be required 
in that case this property atlas.kafka.poll.timeout.ms can be changed 
accordingly


- Nixon


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60597/#review180395
---


On July 14, 2017, 9:52 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60597/
> ---
> 
> (Updated July 14, 2017, 9:52 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1908
> https://issues.apache.org/jira/browse/ATLAS-1908
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch handles old kafka consumer configs values to be added as new 
> configs if new configs are not present , so that Kafka consumer starts 
> gracefully even if new configs are not added explicitly.
> 
> eg. atlas.kafka.auto.commit.enable=false to 
> atlas.kafka.enable.auto.commit=false
> atlas.kafka.auto.offset.reset=smallest to 
> atlas.kafka.auto.offset.reset=earliest
> atlas.kafka.session.timeout.ms=3
> 
> Also added poll.timeout.ms in configs so kafka consumer can be 
> configurable.This involve change in AtlasKafkaConsumer receive interface.
> 
> 
> Diffs
> -
> 
>   distro/src/conf/atlas-application.properties c3213df 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> 9c15243 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
> 366c8a7 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  22e40f9 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 
> 70059cb 
>   
> notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 
> c791d43 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  8324b57 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  0dea0e2 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
>  650ca0a 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> c036cfa 
> 
> 
> Diff: https://reviews.apache.org/r/60597/diff/2/
> 
> 
> Testing
> ---
> 
> Executed UT/IT's using mvn clean install.
> Tested KafkaConsumer with Hive Hook by creating table in hive.
> Atlas starts gracefully and Kafka consumer is up without error when new 
> properties are not added.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 60597: ATLAS-1908 : - Updating old Kafka consumer api properties to reflect change in new KafkaConsumer configs for Atlas

2017-07-14 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60597/
---

(Updated July 14, 2017, 9:52 a.m.)


Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.


Changes
---

Handled review comments from Sarath and Ashutosh.


Bugs: ATLAS-1908
https://issues.apache.org/jira/browse/ATLAS-1908


Repository: atlas


Description
---

This patch handles old kafka consumer configs values to be added as new configs 
if new configs are not present , so that Kafka consumer starts gracefully even 
if new configs are not added explicitly.

eg. atlas.kafka.auto.commit.enable=false to atlas.kafka.enable.auto.commit=false
atlas.kafka.auto.offset.reset=smallest to 
atlas.kafka.auto.offset.reset=earliest
atlas.kafka.session.timeout.ms=3

Also added poll.timeout.ms in configs so kafka consumer can be 
configurable.This involve change in AtlasKafkaConsumer receive interface.


Diffs (updated)
-

  distro/src/conf/atlas-application.properties c3213df 
  notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
9c15243 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
366c8a7 
  
notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
 22e40f9 
  notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 
70059cb 
  notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 
c791d43 
  
notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
 8324b57 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 0dea0e2 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
 650ca0a 
  webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
c036cfa 


Diff: https://reviews.apache.org/r/60597/diff/2/

Changes: https://reviews.apache.org/r/60597/diff/1-2/


Testing
---

Executed UT/IT's using mvn clean install.
Tested KafkaConsumer with Hive Hook by creating table in hive.
Atlas starts gracefully and Kafka consumer is up without error when new 
properties are not added.


Thanks,

Nixon Rodrigues