Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/#review185373
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 14, 2017, 5:15 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62129/
> ---
> 
> (Updated Sept. 14, 2017, 5:15 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2117
> https://issues.apache.org/jira/browse/ATLAS-2117
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddf117fa 
> 
> 
> Diff: https://reviews.apache.org/r/62129/diff/5/
> 
> 
> Testing
> ---
> 
> UI and REST calls working as expected now.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/
---

(Updated Sept. 14, 2017, 5:15 a.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

Removed offending characters logic for a different review and JIRA


Bugs: ATLAS-2117
https://issues.apache.org/jira/browse/ATLAS-2117


Repository: atlas


Description
---

See JIRA


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddf117fa 


Diff: https://reviews.apache.org/r/62129/diff/5/

Changes: https://reviews.apache.org/r/62129/diff/4-5/


Testing
---

UI and REST calls working as expected now.


Thanks,

Apoorv Naik

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Apoorv Naik]

> On Sept. 14, 2017, 2:55 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
> > Lines 232 (patched)
> > 
> >
> > Instead of completely rejecting index-query on detecting NEQ on a 
> > string attribute, consider treating this as a non-indexed attribute (line 
> > #240) - so that index-query can handle other attributes and this attribute 
> > can be handled by graph query or in-memory query.

That's a good idea, I think it's better to move this section in attribute 
processing call and any such violation should mark the attribute to be filtered 
by graph. Will update the review shortly


- Apoorv


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/#review185363
---


On Sept. 14, 2017, 2:32 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62129/
> ---
> 
> (Updated Sept. 14, 2017, 2:32 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2117
> https://issues.apache.org/jira/browse/ATLAS-2117
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddf117fa 
> 
> 
> Diff: https://reviews.apache.org/r/62129/diff/4/
> 
> 
> Testing
> ---
> 
> UI and REST calls working as expected now.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Build failed in Jenkins: Atlas-0.8-IntegrationTests #56

[Apache Jenkins Server]

See 


Changes:

[amestry] ATLAS-2134: Code Improvements.

--
[...truncated 42.69 MB...]
2017-09-14 03:02:07,659 DEBUG - [NotificationHookConsumer thread-0:] ~ Sending 
message for topic ATLAS_ENTITIES: 
{"version":{"version":"1.0.0"},"message":{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"5b02ca08-2938-4ff5-aacd-bd4d692addad","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"name","qualifiedName":"default.tablek8urmyl5gq.name@primary","position":1,"owner":"jenkins","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"6d0adea5-ec43-4ca8-a3ac-4780eee1fe69","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"operationType":"ENTITY_CREATE","traits":[]}}
 (KafkaNotification:218)
2017-09-14 03:02:07,659 DEBUG - [NotificationHookConsumer thread-0:] ~ Sending 
message for topic ATLAS_ENTITIES: 
{"version":{"version":"1.0.0"},"message":{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"b04ab767-b504-4d22-8097-1304a05073d1","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.tablek8urmyl5gq@primary_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"6d0adea5-ec43-4ca8-a3ac-4780eee1fe69","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{},"systemAttributes":{}},"operationType":"ENTITY_CREATE","traits":[]}}
 (KafkaNotification:218)
2017-09-14 03:02:07,660 DEBUG - [NotificationHookConsumer thread-0:] ~ Sending 
message for topic ATLAS_ENTITIES: 
{"version":{"version":"1.0.0"},"message":{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"b62d2fa8-1f41-4cca-849a-cbd9af7c70a9","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"VIRTUAL_VIEW","name":"tablevxoi1vojch","viewExpandedText":"select
 `tablek8urmyl5gq`.`id`, `tablek8urmyl5gq`.`name` from 
`default`.`tablek8URMYl5gq`","createTime":"2017-09-14T02:59:51.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"fae63d45-bece-4340-8584-feb154624851","version":0,"typeName":"hive_db","state":"ACTIVE"},"viewOriginalText":"select
 * from 

Build failed in Jenkins: Atlas-master-UnitTests #72

[Apache Jenkins Server]

See 


Changes:

[amestry] ATLAS-2134: Code Improvements.

--
[...truncated 460.76 KB...]
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-war-plugin:2.6:war (default-war) @ atlas-webapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [atlas-webapp] in 
[
[INFO] Processing overlay [ id org.apache.atlas:atlas-dashboardv2]
[INFO] Processing war project
[INFO] Copying webapp resources 
[
[INFO] Building jar: 

[INFO] Webapp assembled in [1503 msecs]
[INFO] Building war: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
atlas-webapp ---
[INFO] 
[INFO] --- enunciate-maven-plugin:2.8.0:docs (default) @ atlas-webapp ---
[INFO] [ENUNCIATE] Skipping enunciate per configuration.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-webapp ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[WARNING] file.encoding cannot be set as system property, use 
-Dfile.encoding=... instead

---
 T E S T S
---
Running org.apache.atlas.notification.NotificationHookConsumerTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.461 sec - in 
org.apache.atlas.notification.NotificationHookConsumerTest
Running org.apache.atlas.notification.NotificationEntityChangeListenerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.922 sec - in 
org.apache.atlas.notification.NotificationEntityChangeListenerTest
Running org.apache.atlas.notification.NotificationHookConsumerKafkaTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.958 sec - in 
org.apache.atlas.notification.NotificationHookConsumerKafkaTest
Running org.apache.atlas.notification.AdaptiveWaiterTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.937 sec - in 
org.apache.atlas.notification.AdaptiveWaiterTest
Running org.apache.atlas.web.filters.AuditFilterTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.283 sec - in 
org.apache.atlas.web.filters.AuditFilterTest
Running org.apache.atlas.web.filters.AtlasAuthenticationKerberosFilterTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.332 sec - in 
org.apache.atlas.web.filters.AtlasAuthenticationKerberosFilterTest
Running org.apache.atlas.web.filters.ActiveServerFilterTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.043 sec - in 
org.apache.atlas.web.filters.ActiveServerFilterTest
Running org.apache.atlas.web.filters.AtlasCSRFPreventionFilterTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.169 sec - in 
org.apache.atlas.web.filters.AtlasCSRFPreventionFilterTest
Running org.apache.atlas.web.service.ActiveInstanceStateTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.001 sec - in 
org.apache.atlas.web.service.ActiveInstanceStateTest
Running org.apache.atlas.web.service.ActiveInstanceElectorServiceTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.27 sec - in 
org.apache.atlas.web.service.ActiveInstanceElectorServiceTest
Running org.apache.atlas.web.service.AtlasZookeeperSecurityPropertiesTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.387 sec - in 
org.apache.atlas.web.service.AtlasZookeeperSecurityPropertiesTest
Running org.apache.atlas.web.service.SecureEmbeddedServerTest
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 35.091 sec <<< 
FAILURE! - in org.apache.atlas.web.service.SecureEmbeddedServerTest

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/#review185363
---




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Lines 232 (patched)


Instead of completely rejecting index-query on detecting NEQ on a string 
attribute, consider treating this as a non-indexed attribute (line #240) - so 
that index-query can handle other attributes and this attribute can be handled 
by graph query or in-memory query.


- Madhan Neethiraj


On Sept. 14, 2017, 2:32 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62129/
> ---
> 
> (Updated Sept. 14, 2017, 2:32 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2117
> https://issues.apache.org/jira/browse/ATLAS-2117
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddf117fa 
> 
> 
> Diff: https://reviews.apache.org/r/62129/diff/4/
> 
> 
> Testing
> ---
> 
> UI and REST calls working as expected now.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/
---

(Updated Sept. 14, 2017, 2:32 a.m.)


Review request for atlas and Madhan Neethiraj.


Bugs: ATLAS-2117
https://issues.apache.org/jira/browse/ATLAS-2117


Repository: atlas


Description
---

See JIRA


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddf117fa 


Diff: https://reviews.apache.org/r/62129/diff/4/

Changes: https://reviews.apache.org/r/62129/diff/3-4/


Testing
---

UI and REST calls working as expected now.


Thanks,

Apoorv Naik

[jira] [Updated] (ATLAS-2117) Basic search issues due to Titan Solr schema

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2117:
---
Attachment: 0001-ATLAS-2117-Work-around-for-basic-search-due-to-Titan.patch

> Basic search issues due to Titan Solr schema
> 
>
> Key: ATLAS-2117
> URL: https://issues.apache.org/jira/browse/ATLAS-2117
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>
> Attachments: 
> 0001-ATLAS-2117-Work-around-for-basic-search-due-to-Titan.patch
>
>
> When using Solr as indexing backend, the tokenization of the string is 
> performed using the StandardTokenizerFactory which treats punctuations and 
> special characters as delimiters which results in the more indexed terms 
> being associated with the associated vertex (document)
> Also there's a LowercaseFilterFactory which makes lookup case insensitive.
> This schema design doesn't work well for the current basic search enhancement 
> (ATLAS-1880) causing a lot of false positives/negatives when querying the 
> index.
> The workaround/hack for this is to do an in-memory filtering when such schema 
> violations are found or push the entire attribute query down to the graph 
> which might be in-efficient and memory intensive. (Current JIRA will track 
> this)
> Correct solution would be to re-index the existing data with a schema change 
> and not use the mentioned code workarounds for better performance of the 
> search. (Should be taken up in separate JIRA)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2117) Basic search issues due to Titan Solr schema

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2117:
---
Attachment: (was: 
0001-ATLAS-2117-Work-around-for-basic-search-due-to-Titan.patch)

> Basic search issues due to Titan Solr schema
> 
>
> Key: ATLAS-2117
> URL: https://issues.apache.org/jira/browse/ATLAS-2117
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>
> Attachments: 
> 0001-ATLAS-2117-Work-around-for-basic-search-due-to-Titan.patch
>
>
> When using Solr as indexing backend, the tokenization of the string is 
> performed using the StandardTokenizerFactory which treats punctuations and 
> special characters as delimiters which results in the more indexed terms 
> being associated with the associated vertex (document)
> Also there's a LowercaseFilterFactory which makes lookup case insensitive.
> This schema design doesn't work well for the current basic search enhancement 
> (ATLAS-1880) causing a lot of false positives/negatives when querying the 
> index.
> The workaround/hack for this is to do an in-memory filtering when such schema 
> violations are found or push the entire attribute query down to the graph 
> which might be in-efficient and memory intensive. (Current JIRA will track 
> this)
> Correct solution would be to re-index the existing data with a schema change 
> and not use the mentioned code workarounds for better performance of the 
> search. (Should be taken up in separate JIRA)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2134:
---
Attachment: (was: ATLAS-2134-CodeImprovements.patch)

> Code Improvement To Follow Best Practices
> -
>
> Key: ATLAS-2134
> URL: https://issues.apache.org/jira/browse/ATLAS-2134
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
>
> Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2134:
---
Attachment: ATLAS-2134-CodeImprovements.patch

> Code Improvement To Follow Best Practices
> -
>
> Key: ATLAS-2134
> URL: https://issues.apache.org/jira/browse/ATLAS-2134
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-2134-CodeImprovements.patch
>
>
> Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2134:
---
Attachment: (was: ATLAS-2134-CodeImprovements.patch)

> Code Improvement To Follow Best Practices
> -
>
> Key: ATLAS-2134
> URL: https://issues.apache.org/jira/browse/ATLAS-2134
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
>
> Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Draft Board Report for Apache Atlas: September-2017

[Madhan Neethiraj]

Atlas team,

 

Please review the draft board report below and send your feedback/comments,

if any.

 

Thanks,

Madhan

 

 

 

## Description:

  Apache Atlas is a scalable and extensible set of core foundational governance

  services that enables enterprises to effectively and efficiently meet their

  compliance requirements within Hadoop and allows integration with the

  complete enterprise data ecosystem

 

## Issues:

  There are no issues requiring board attention at this time.

 

## Activity:

  - Released 0.8.1, the first release after graduation

  - Completed backend work on relationships as first class type

  - Working on new features like tag-propagation, business catalog,

    Open Metadata & Governance, Virtual Data Connector,

    Automated Discovery

 

## Health report:

  - Added 1 new committer in last month

  - No new contributors added in last month

 

## PMC changes:

  - Currently 33 PMC members

  - No new PMC members added in last month

  - Last PMC member addition was on 6/21/2017

 

## Committer base changes:

  - Currently 36 committers

  - 1 new committer added in last month, on 8/15/2017

  - Last addition to committer role was Mandy Chessell

 

## Releases:

 0.8.1    was released on 08/29/2017

 0.8-incubating   was released on 03/16/2017

 0.7.1-incubating was released on 01/26/2017

 0.7-incubating   was released on 07/09/2016

 0.6-incubating   was released on 12/31/2015

 0.5-incubating   was released on 07/11/2015

 

 

 

 

[jira] [Updated] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2134:
---
Attachment: (was: ATLAS-2134-ZipSource-printStack.patch)

> Code Improvement To Follow Best Practices
> -
>
> Key: ATLAS-2134
> URL: https://issues.apache.org/jira/browse/ATLAS-2134
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-2134-CodeImprovements.patch
>
>
> Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2134:
---
Attachment: ATLAS-2134-CodeImprovements.patch

> Code Improvement To Follow Best Practices
> -
>
> Key: ATLAS-2134
> URL: https://issues.apache.org/jira/browse/ATLAS-2134
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-2134-CodeImprovements.patch
>
>
> Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: Atlas-0.8-IntegrationTests #55

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2115: Basic search updates to fix performance regression

--
[...truncated 509.16 KB...]
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/structdef/name/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/structdef/guid/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/classificationdef/name/blah HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/classificationdef/guid/blah HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/entitydef/name/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/entitydef/guid/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/typedefs/?supertype=AgjWQbMSoDW=CLASS HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "GET 
/api/atlas/v2/types/typedefs/?notsupertype=BtHKUtBRt3D=AgjWQbMSoDW=CLASS
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:08 +] "PUT /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "GET 
/api/atlas/v2/types/typedefs/?type=ENTITY HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "POST /api/atlas/types HTTP/1.1" 409 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "GET /api/atlas/types/blah HTTP/1.1" 
404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "GET 
/api/atlas/v2/types/typedef/name/class_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "GET 
/api/atlas/v2/types/typedef/name/secure_trait HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "GET 
/api/atlas/v2/types/typedef/name/pii_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:09 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "GET 
/api/atlas/v2/types/typedef/name/ssn_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "GET 
/api/atlas/v2/types/typedef/name/salary_trait HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "GET 
/api/atlas/v2/types/typedef/name/sox_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "GET /api/atlas/types?type=TRAIT 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "GET 
/api/atlas/v2/types/typedef/name/AvpdgzRAgWF HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "GET 
/api/atlas/v2/types/typedef/name/A1hmngPNZtd9 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:10 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:11 +] "GET 
/api/atlas/v2/types/typedef/name/BmJR4VgEFtr HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:11 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:11 +] "GET 
/api/atlas/v2/types/typedef/name/CIqyZ7EnpQr HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:11 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:11 +] "GET 
/api/atlas/types?type=CLASS=AvpdgzRAgWF=BmJR4VgEFtr 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:11 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:12 +] "PUT /api/atlas/types HTTP/1.1" 200 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:53:12 +] "GET /api/atlas/types/hlltLOryXu 
HTTP/1.1" 200 - "-" 

Build failed in Jenkins: Atlas-master-UnitTests #71

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2115: Basic search updates to fix performance regression

--
[...truncated 454.11 KB...]
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-war-plugin:2.6:war (default-war) @ atlas-webapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [atlas-webapp] in 
[
[INFO] Processing overlay [ id org.apache.atlas:atlas-dashboardv2]
[INFO] Processing war project
[INFO] Copying webapp resources 
[
[INFO] Building jar: 

[INFO] Webapp assembled in [1605 msecs]
[INFO] Building war: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
atlas-webapp ---
[INFO] 
[INFO] --- enunciate-maven-plugin:2.8.0:docs (default) @ atlas-webapp ---
[INFO] [ENUNCIATE] Skipping enunciate per configuration.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-webapp ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[WARNING] file.encoding cannot be set as system property, use 
-Dfile.encoding=... instead

---
 T E S T S
---
Running org.apache.atlas.notification.NotificationHookConsumerKafkaTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.722 sec - in 
org.apache.atlas.notification.NotificationHookConsumerKafkaTest
Running org.apache.atlas.notification.NotificationEntityChangeListenerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.919 sec - in 
org.apache.atlas.notification.NotificationEntityChangeListenerTest
Running org.apache.atlas.notification.NotificationHookConsumerTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.444 sec - in 
org.apache.atlas.notification.NotificationHookConsumerTest
Running org.apache.atlas.notification.AdaptiveWaiterTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.936 sec - in 
org.apache.atlas.notification.AdaptiveWaiterTest
Running org.apache.atlas.web.filters.AtlasAuthenticationKerberosFilterTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.33 sec - in 
org.apache.atlas.web.filters.AtlasAuthenticationKerberosFilterTest
Running org.apache.atlas.web.filters.ActiveServerFilterTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.06 sec - in 
org.apache.atlas.web.filters.ActiveServerFilterTest
Running org.apache.atlas.web.filters.AuditFilterTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.279 sec - in 
org.apache.atlas.web.filters.AuditFilterTest
Running org.apache.atlas.web.filters.AtlasCSRFPreventionFilterTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.18 sec - in 
org.apache.atlas.web.filters.AtlasCSRFPreventionFilterTest
Running org.apache.atlas.web.service.SecureEmbeddedServerTest
Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 34.55 sec <<< 
FAILURE! - in org.apache.atlas.web.service.SecureEmbeddedServerTest
testServerConfiguredUsingCredentialProvider(org.apache.atlas.web.service.SecureEmbeddedServerTest)
  Time elapsed: 21.252 sec  <<< FAILURE!
java.lang.AssertionError: War deploy failed
at org.testng.Assert.fail(Assert.java:89)
at org.testng.Assert.failNotEquals(Assert.java:489)
at org.testng.Assert.assertEquals(Assert.java:118)
at org.testng.Assert.assertEquals(Assert.java:365)
at org.testng.Assert.assertEquals(Assert.java:375)
at 
org.apache.atlas.web.service.SecureEmbeddedServerTest.testServerConfiguredUsingCredentialProvider(SecureEmbeddedServerTest.java:77)

Running 

Build failed in Jenkins: Atlas-master-IntegrationTests #93

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2115: Basic search updates to fix performance regression

--
[...truncated 442.60 KB...]
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/pci HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/sox HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/sec HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/finance HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/FacttMsELcUzP7 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/ETLqCQtXKIlCE HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/DimensionT6FqaNmfQB HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/MetricJ6J05RIzMW HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/hive_db_v1 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/hive_column_v1 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/hive_table_v1 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "GET 
/api/atlas/v2/types/typedef/name/hive_process_v1 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:18 +] "POST /api/atlas/types HTTP/1.1" 400 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/tableType HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/serdeType HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/classification HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/pii HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/phi HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/pci HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/sox HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/sec HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/finance HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/Factd5CbpJw81m HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/ETLB3vRYn38Ot HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/DimensiongytKBoBdkf HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/MetricxEQQSSl85o HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/hive_db_v1 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/hive_column_v1 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/hive_table_v1 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/typedef/name/hive_process_v1 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "POST /api/atlas/types HTTP/1.1" 400 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 409 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/enumdef/name/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/enumdef/guid/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/structdef/name/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] "GET 
/api/atlas/v2/types/structdef/guid/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:20:43:19 +] 

Re: Review Request 62127: ATLAS-2118: Basic search contains issue

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62127/
---

(Updated Sept. 13, 2017, 8:11 p.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

Rebase with latest master


Bugs: ATLAS-2118
https://issues.apache.org/jira/browse/ATLAS-2118


Repository: atlas


Description
---

Any reserved regex character causes the contains check to throw an exception 
from the titan graph query layer.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 1d4815df 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddf117fa 


Diff: https://reviews.apache.org/r/62127/diff/2/

Changes: https://reviews.apache.org/r/62127/diff/1-2/


Testing
---

UI and REST calls work as expected now


Thanks,

Apoorv Naik

[jira] [Updated] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2134:
---
Attachment: ATLAS-2134-ZipSource-printStack.patch

> Code Improvement To Follow Best Practices
> -
>
> Key: ATLAS-2134
> URL: https://issues.apache.org/jira/browse/ATLAS-2134
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-2134-ZipSource-printStack.patch
>
>
> Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2134) Code Improvement To Follow Best Practices

[Ashutosh Mestry (JIRA)]

Ashutosh Mestry created ATLAS-2134:
--

 Summary: Code Improvement To Follow Best Practices
 Key: ATLAS-2134
 URL: https://issues.apache.org/jira/browse/ATLAS-2134
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.8-incubating
Reporter: Ashutosh Mestry
Assignee: Ashutosh Mestry
 Fix For: trunk


Code Improvement To Follow Best Practices



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (ATLAS-2115) Regression: Basic search performance

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-2115.
-
   Resolution: Fixed
Fix Version/s: (was: 0.8.1-incubating)
   (was: 0.8-incubating)
   (was: trunk)
   0.8.2
   0.9-incubating

Committed to the following branches:
 - master: http://git-wip-us.apache.org/repos/asf/atlas/commit/1dc7f549
 - branch-0.8: http://git-wip-us.apache.org/repos/asf/atlas/commit/221e829f

> Regression: Basic search performance
> 
>
> Key: ATLAS-2115
> URL: https://issues.apache.org/jira/browse/ATLAS-2115
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.8.1-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.9-incubating, 0.8.2
>
>
> When searching using a tag without any attributes the call timings were 
> doubled (worst case tripled) regardless of the offset and limit. This patch 
> introduces a new code path which allows faster retrieval of vertices for the 
> mentioned scenario. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2118) Basic search - Contains clause issues

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2118:
---
Attachment: 0001-ATLAS-2118-Issues-with-contains-in-Basic-search.patch

> Basic search - Contains clause issues
> -
>
> Key: ATLAS-2118
> URL: https://issues.apache.org/jira/browse/ATLAS-2118
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>
> Attachments: 
> 0001-ATLAS-2118-Issues-with-contains-in-Basic-search.patch
>
>
> If the contains clause attribute value contains any of the special regex 
> chars like 
> "+", "|", "(", "{", "[", "*", "?", "$", "/", "^"
> then the graph query fails with Invalid pattern exception
> Solution: Escape all occurrences of the special characters 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2117) Basic search issues due to Titan Solr schema

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2117:
---
Attachment: 0001-ATLAS-2117-Work-around-for-basic-search-due-to-Titan.patch

> Basic search issues due to Titan Solr schema
> 
>
> Key: ATLAS-2117
> URL: https://issues.apache.org/jira/browse/ATLAS-2117
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>
> Attachments: 
> 0001-ATLAS-2117-Work-around-for-basic-search-due-to-Titan.patch
>
>
> When using Solr as indexing backend, the tokenization of the string is 
> performed using the StandardTokenizerFactory which treats punctuations and 
> special characters as delimiters which results in the more indexed terms 
> being associated with the associated vertex (document)
> Also there's a LowercaseFilterFactory which makes lookup case insensitive.
> This schema design doesn't work well for the current basic search enhancement 
> (ATLAS-1880) causing a lot of false positives/negatives when querying the 
> index.
> The workaround/hack for this is to do an in-memory filtering when such schema 
> violations are found or push the entire attribute query down to the graph 
> which might be in-efficient and memory intensive. (Current JIRA will track 
> this)
> Correct solution would be to re-index the existing data with a schema change 
> and not use the mentioned code workarounds for better performance of the 
> search. (Should be taken up in separate JIRA)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/#review185330
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 13, 2017, 7:26 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62110/
> ---
> 
> (Updated Sept. 13, 2017, 7:26 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2115
> https://issues.apache.org/jira/browse/ATLAS-2115
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1880 introduced regression in call timings for the basic search GET 
> implementation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> 972c11e3 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  0daab030 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  a4a638af 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> fc973e6f 
> 
> 
> Diff: https://reviews.apache.org/r/62110/diff/5/
> 
> 
> Testing
> ---
> 
> Tested via scripts and network call timing in Chrome/Firefox.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/
---

(Updated Sept. 13, 2017, 7:26 p.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

ADdressed review comments


Bugs: ATLAS-2115
https://issues.apache.org/jira/browse/ATLAS-2115


Repository: atlas


Description
---

ATLAS-1880 introduced regression in call timings for the basic search GET 
implementation


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
972c11e3 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 0daab030 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
a4a638af 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
d5e39236 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
fc973e6f 


Diff: https://reviews.apache.org/r/62110/diff/5/

Changes: https://reviews.apache.org/r/62110/diff/4-5/


Testing
---

Tested via scripts and network call timing in Chrome/Firefox.


Thanks,

Apoorv Naik

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/#review185312
---




repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
Lines 286 (patched)


This comment doesn't look right. Please review and update.



repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
Lines 67 (patched)


Since typeName is a single-value, 'in' predicate seems appropriate. Please 
review.



repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java
Line 325 (original), 400 (patched)


this should be:
 return ((Collection)attrValue).contains(value);



repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java
Lines 526 (patched)


this should be:
 return ((Collection)value).contains(attrVal);


- Madhan Neethiraj


On Sept. 13, 2017, 5:28 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62110/
> ---
> 
> (Updated Sept. 13, 2017, 5:28 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2115
> https://issues.apache.org/jira/browse/ATLAS-2115
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1880 introduced regression in call timings for the basic search GET 
> implementation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> 972c11e3 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  0daab030 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  a4a638af 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> fc973e6f 
> 
> 
> Diff: https://reviews.apache.org/r/62110/diff/4/
> 
> 
> Testing
> ---
> 
> Tested via scripts and network call timing in Chrome/Firefox.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/
---

(Updated Sept. 13, 2017, 5:28 p.m.)


Review request for atlas and Madhan Neethiraj.


Changes
---

Added a BooleanPredicate alongwith new supported Operators (contains_any, 
contains_all)


Bugs: ATLAS-2115
https://issues.apache.org/jira/browse/ATLAS-2115


Repository: atlas


Description
---

ATLAS-1880 introduced regression in call timings for the basic search GET 
implementation


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
972c11e3 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 0daab030 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
a4a638af 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
d5e39236 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
fc973e6f 


Diff: https://reviews.apache.org/r/62110/diff/4/

Changes: https://reviews.apache.org/r/62110/diff/3-4/


Testing
---

Tested via scripts and network call timing in Chrome/Firefox.


Thanks,

Apoorv Naik

[jira] [Assigned] (ATLAS-2132) Error code during invalid file path/unreadable file provided during import

[Ashutosh Mestry (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry reassigned ATLAS-2132:
--

Assignee: Ashutosh Mestry

> Error code during invalid file path/unreadable file provided during import
> --
>
> Key: ATLAS-2132
> URL: https://issues.apache.org/jira/browse/ATLAS-2132
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Minor
>
> When firing import command using 
> {code}
> /api/atlas/admin/importfile
> {code}
> when file provided in the import_options.json doesn't have read permission or 
> file is not present  , Atlas throws 500 Internal server error but with proper 
> error message. 
> Example  :
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
>  invalid parameters: /exportimport/db5.zip: file not found"}
> {code}
> {code}
> {"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
>  java.io.IOException: File '/exportimport/db6.zip' cannot be read"}
> {code}
> Expected that Atlas would throw 400 Bad Request instead of 500 Internal 
> Server error.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61940: ATLAS-2090 : UI - Cache busting for static content (css, js)

[Ashutosh Mestry]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61940/#review185305
---


Ship it!




Ship It!

- Ashutosh Mestry


On Sept. 13, 2017, 2:11 p.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61940/
> ---
> 
> (Updated Sept. 13, 2017, 2:11 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2090
> https://issues.apache.org/jira/browse/ATLAS-2090
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Grunt task is updated to handle Cache busting issue, so now after every 
> successful build grunt task will add new time as the query string for static 
> content i.e .CSS and .JS
> 
> 
> Diffs
> -
> 
>   dashboardv2/gruntfile.js 900ee40 
>   dashboardv2/package.json 6068c79 
>   dashboardv2/public/index.html f572049 
>   dashboardv2/public/index.html.tpl PRE-CREATION 
>   dashboardv2/public/js/main.js 63e9abc 
> 
> 
> Diff: https://reviews.apache.org/r/61940/diff/2/
> 
> 
> Testing
> ---
> 
> Build the atlas code and checked the query string for static content i.e.CSS 
> and .JS, its working as expected.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[Madhan Neethiraj]

> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > 
> >
> > can we include descriptions for existing attributes and 
> > relationshipEndDefs for out of the box models as part of this JIRA?
> 
> David Radley wrote:
> I think it is better to separate the putting values into the models into 
> separate Jiras. We can then use those Jiras to review the description content 
> rather than the mechanism to introduce description.
> 
> Madhan Neethiraj wrote:
> I agree. However, I would suggest adding 'Map aliases' 
> field to struct-def and attribute-def - to enable alternate names for these 
> to be stored along with entity-def, classification-def and attribute-def. 
> This should be a trivial update and can be taken up along with addition of 
> 'description' field.
> 
> David Radley wrote:
> I am unsure what the keys and values might be for the aliases map you are 
> proposing. Are you thinking that the aliases would be instance aliases or 
> type aliases? 
> 
> We are thinking of adding external type identifications 
> (ExternalTypeIdDefs) as a new array associated with attributes, structures 
> (and the otehr defs). We are thinking the elements would be specified in the 
> typeDefs in the form:
> 
> ExternalTypeIdDefs [
> {   "scheme":"O-DEF",
> "value":"123-456",
> "description":"Relational Database",
> "ResponsibleOrg":"Open Group",
> "ReferenceDocumentationURL":"http://xxx.yyy.zzz;
> },
> {
> ...
> }
> ]
> 
> Scheme and value would be manditory.
> 
> One way to handle type aliases would be using the above mechanism: 
> {   "scheme":"alias",
> "value":"RDB",
> "description":"short hand relational database key for consumer A",
> "ResponsibleOrg":"Consumer A organisation",
> "ReferenceDocumentationURL":"http://aaa.bbb.bbb;
> },
> 
> or
> 
> {   "scheme":"alias",
> "value":"RDB"
> },
> 
> 
> This would allow these external type identifier metadata values to be 
> visible in the json models.
> 
> I was thinking of doing this piece separately in another Jira. The 
> addition of the code as-is in the Jira will in enable me to add desciptions 
> to the attributes / relationship ends in the new models. For the existing 
> Hadoop models - it might be worth having someone more expert in Hadoop 
> filling in those descriptions.
> 
> David Radley wrote:
> One other thought - if the aliases map are for instances - I wonder if 
> they could be indexed. I assume they could be specified in a search query.
> 
> Madhan Neethiraj wrote:
> We discussed about an entity-type or an attribute in Atlas being referred 
> with a different name in other meta-data standards. The 'alias' field I 
> suggested above is to capture these names along with entity-def/attribute-def 
> in Atlas. I assume it will be enough to capure only the name used in other 
> standards; if more richer information is available, how likely it is to be 
> able to capture all the details available in various standards? To keep it 
> simple, we can start with only capturing the names used in other meta-data 
> standards.
> 
> David Radley wrote:
> Hi Madhan,
> Thanks for your quick reply. We did start with name and value pair, but 
> felt the additional fields could give real value, for those that use them.  
> 
> To answer " how likely it is to be able to capture all the details 
> available in various standards". I think that for the O-DEF content we are 
> thinking about adding to the models, we could fill out all this content. If 
> other standards do not want to or have this information, then they can omit 
> the optional fields. The advantage of the "ResponsibleOrg" and 
> "ReferenceDocumentationURL" is that they leave no doubt as to the author of 
> these names, which is great for auditing and maybe even lineage 
> visualizations. Just calling this alias may imply this is a display name - 
> but for the use case I was thinking of, it is actually an external identifier 
> in another metadata standard.  
> 
> I hope this approach keeps it simple for people wanting to add name value 
> pairs, also allowing us to have the values in the model files (a very easy 
> proof point to show we can work with O-DEF) and also gives us the ability to 
> add the extra provenance information. Lets discuss further on another Jira if 
> we need to.
> 
> David Radley wrote:
> Raised Jira https://issues.apache.org/jira/browse/ATLAS-2133 to track the 
> alias / ExternalTypeIdDefs work

ok. Let's discuss the 'external-id' details via ATLAS-2133.


- Madhan


---
This is an automatically generated 

[jira] [Updated] (ATLAS-2090) UI - Cache busting for static content (css, js)

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-2090:
---
Attachment: ATLAS-2090.1.patch

> UI - Cache busting for static content (css, js)
> ---
>
> Key: ATLAS-2090
> URL: https://issues.apache.org/jira/browse/ATLAS-2090
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.9-incubating
>Reporter: Nixon Rodrigues
>Assignee: Keval Bhatt
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-2090.1.patch, ATLAS-2090.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2133) Add ExternalTypeIdDefs to every Def element

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-2133:

Description: 
Adding external type identifications (ExternalTypeIdDefs) as a new array 
associated with attributes, structures (and the otehr defs). We are thinking 
the elements would be specified in the typeDefs in the form:

ExternalTypeIdDefs [
{   "scheme":"O-DEF",
"value":"123-456",
"description":"Relational Database",
"ResponsibleOrg":"Open Group",
"ReferenceDocumentationURL":"http://xxx.yyy.zzz;
},
{
...
}
]

Scheme and value would be manditory.

One way to handle type aliases would be using the above mechanism: 
{   "scheme":"alias",
"value":"RDB",
"description":"short hand relational database key for consumer A",
"ResponsibleOrg":"Consumer A organisation",
"ReferenceDocumentationURL":"http://aaa.bbb.bbb;
},

or

{   "scheme":"alias",
"value":"RDB"
},

There has been some discussion on this in the comments associated with 
https://reviews.apache.org/r/62244/



  was:
Adding external type identifications (ExternalTypeIdDefs) as a new array 
associated with attributes, structures (and the otehr defs). We are thinking 
the elements would be specified in the typeDefs in the form:

ExternalTypeIdDefs [
{   "scheme":"O-DEF",
"value":"123-456",
"description":"Relational Database",
"ResponsibleOrg":"Open Group",
"ReferenceDocumentationURL":"http://xxx.yyy.zzz;
},
{
...
}
]

Scheme and value would be manditory.

One way to handle type aliases would be using the above mechanism: 
{   "scheme":"alias",
"value":"RDB",
"description":"short hand relational database key for consumer A",
"ResponsibleOrg":"Consumer A organisation",
"ReferenceDocumentationURL":"http://aaa.bbb.bbb;
},

or

{   "scheme":"alias",
"value":"RDB"
},


> Add ExternalTypeIdDefs to every Def element
> ---
>
> Key: ATLAS-2133
> URL: https://issues.apache.org/jira/browse/ATLAS-2133
> Project: Atlas
>  Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
>
> Adding external type identifications (ExternalTypeIdDefs) as a new array 
> associated with attributes, structures (and the otehr defs). We are thinking 
> the elements would be specified in the typeDefs in the form:
> ExternalTypeIdDefs [
> {   "scheme":"O-DEF",
> "value":"123-456",
> "description":"Relational Database",
> "ResponsibleOrg":"Open Group",
> "ReferenceDocumentationURL":"http://xxx.yyy.zzz;
> },
> {
> ...
> }
> ]
> Scheme and value would be manditory.
> One way to handle type aliases would be using the above mechanism: 
> {   "scheme":"alias",
> "value":"RDB",
> "description":"short hand relational database key for consumer A",
> "ResponsibleOrg":"Consumer A organisation",
> "ReferenceDocumentationURL":"http://aaa.bbb.bbb;
> },
> or
> {   "scheme":"alias",
> "value":"RDB"
> },
> There has been some discussion on this in the comments associated with 
> https://reviews.apache.org/r/62244/



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[David Radley]

> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > 
> >
> > can we include descriptions for existing attributes and 
> > relationshipEndDefs for out of the box models as part of this JIRA?
> 
> David Radley wrote:
> I think it is better to separate the putting values into the models into 
> separate Jiras. We can then use those Jiras to review the description content 
> rather than the mechanism to introduce description.
> 
> Madhan Neethiraj wrote:
> I agree. However, I would suggest adding 'Map aliases' 
> field to struct-def and attribute-def - to enable alternate names for these 
> to be stored along with entity-def, classification-def and attribute-def. 
> This should be a trivial update and can be taken up along with addition of 
> 'description' field.
> 
> David Radley wrote:
> I am unsure what the keys and values might be for the aliases map you are 
> proposing. Are you thinking that the aliases would be instance aliases or 
> type aliases? 
> 
> We are thinking of adding external type identifications 
> (ExternalTypeIdDefs) as a new array associated with attributes, structures 
> (and the otehr defs). We are thinking the elements would be specified in the 
> typeDefs in the form:
> 
> ExternalTypeIdDefs [
> {   "scheme":"O-DEF",
> "value":"123-456",
> "description":"Relational Database",
> "ResponsibleOrg":"Open Group",
> "ReferenceDocumentationURL":"http://xxx.yyy.zzz;
> },
> {
> ...
> }
> ]
> 
> Scheme and value would be manditory.
> 
> One way to handle type aliases would be using the above mechanism: 
> {   "scheme":"alias",
> "value":"RDB",
> "description":"short hand relational database key for consumer A",
> "ResponsibleOrg":"Consumer A organisation",
> "ReferenceDocumentationURL":"http://aaa.bbb.bbb;
> },
> 
> or
> 
> {   "scheme":"alias",
> "value":"RDB"
> },
> 
> 
> This would allow these external type identifier metadata values to be 
> visible in the json models.
> 
> I was thinking of doing this piece separately in another Jira. The 
> addition of the code as-is in the Jira will in enable me to add desciptions 
> to the attributes / relationship ends in the new models. For the existing 
> Hadoop models - it might be worth having someone more expert in Hadoop 
> filling in those descriptions.
> 
> David Radley wrote:
> One other thought - if the aliases map are for instances - I wonder if 
> they could be indexed. I assume they could be specified in a search query.
> 
> Madhan Neethiraj wrote:
> We discussed about an entity-type or an attribute in Atlas being referred 
> with a different name in other meta-data standards. The 'alias' field I 
> suggested above is to capture these names along with entity-def/attribute-def 
> in Atlas. I assume it will be enough to capure only the name used in other 
> standards; if more richer information is available, how likely it is to be 
> able to capture all the details available in various standards? To keep it 
> simple, we can start with only capturing the names used in other meta-data 
> standards.
> 
> David Radley wrote:
> Hi Madhan,
> Thanks for your quick reply. We did start with name and value pair, but 
> felt the additional fields could give real value, for those that use them.  
> 
> To answer " how likely it is to be able to capture all the details 
> available in various standards". I think that for the O-DEF content we are 
> thinking about adding to the models, we could fill out all this content. If 
> other standards do not want to or have this information, then they can omit 
> the optional fields. The advantage of the "ResponsibleOrg" and 
> "ReferenceDocumentationURL" is that they leave no doubt as to the author of 
> these names, which is great for auditing and maybe even lineage 
> visualizations. Just calling this alias may imply this is a display name - 
> but for the use case I was thinking of, it is actually an external identifier 
> in another metadata standard.  
> 
> I hope this approach keeps it simple for people wanting to add name value 
> pairs, also allowing us to have the values in the model files (a very easy 
> proof point to show we can work with O-DEF) and also gives us the ability to 
> add the extra provenance information. Lets discuss further on another Jira if 
> we need to.

Raised Jira https://issues.apache.org/jira/browse/ATLAS-2133 to track the alias 
/ ExternalTypeIdDefs work


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185223

[jira] [Created] (ATLAS-2133) Add ExternalTypeIdDefs to every Def element

[David Radley (JIRA)]

David Radley created ATLAS-2133:
---

 Summary: Add ExternalTypeIdDefs to every Def element
 Key: ATLAS-2133
 URL: https://issues.apache.org/jira/browse/ATLAS-2133
 Project: Atlas
  Issue Type: Bug
Reporter: David Radley
Assignee: David Radley


Adding external type identifications (ExternalTypeIdDefs) as a new array 
associated with attributes, structures (and the otehr defs). We are thinking 
the elements would be specified in the typeDefs in the form:

ExternalTypeIdDefs [
{   "scheme":"O-DEF",
"value":"123-456",
"description":"Relational Database",
"ResponsibleOrg":"Open Group",
"ReferenceDocumentationURL":"http://xxx.yyy.zzz;
},
{
...
}
]

Scheme and value would be manditory.

One way to handle type aliases would be using the above mechanism: 
{   "scheme":"alias",
"value":"RDB",
"description":"short hand relational database key for consumer A",
"ResponsibleOrg":"Consumer A organisation",
"ReferenceDocumentationURL":"http://aaa.bbb.bbb;
},

or

{   "scheme":"alias",
"value":"RDB"
},



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Reopened] (ATLAS-2111) UI: Select query is not able to render the search table in Advance search

[Keval Bhatt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt reopened ATLAS-2111:


> UI:  Select query is not able to render the search table in Advance search
> --
>
> Key: ATLAS-2111
> URL: https://issues.apache.org/jira/browse/ATLAS-2111
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.9-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-2111.patch
>
>
> Steps to reproduce.
> * Run quick_start and generate few test data.
> * Go to advance search selection
> * now search for the DB where name="Reporting" select name, owner.
> * Table will show empty even though API has data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2029) Enhance classificationDef create to include a list of entities it supports

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-2029:

Attachment: ATLAS2029-5.patch

> Enhance classificationDef create to include a list of entities it supports
> --
>
> Key: ATLAS-2029
> URL: https://issues.apache.org/jira/browse/ATLAS-2029
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: David Radley
>Assignee: David Radley
>  Labels: VirtualDataConnector
> Attachments: ATLAS-2029-1.patch, ATLAS-2029-2.patch, 
> ATLAS-2029-3.patch, ATLAS2029-4.patch, ATLAS2029-5.patch, ATLAS-2029.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61526: ATLAS-2029: Restrict entities, classifications can be applied to

[David Radley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61526/
---

(Updated Sept. 13, 2017, 1:38 p.m.)


Review request for atlas, Graham Wallis, Madhan Neethiraj, and Sarath 
Subramanian.


Repository: atlas


Description
---

ATLAS-2029: Restrict entities, classifications can be applied to


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
f529739d15fec0febe3f33e2bc5418e999eacd84 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasClassificationDef.java 
eeaf71413a56c08db8170fd3323b8e8245ae44fe 
  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
cc3e45ed2f059ad0c5731dc1da7e592d139c3e7a 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
28215fd2aa33ec8011f6900b68c672b685053e7a 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
ab063e23e355f74668af389b97f7da03b2a8f90f 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 
fc65af057255b4c17378080ee4fb7cbfc780c3fe 
  intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 
e1ca889fba1540015850a57232936abad8fd6f37 
  intg/src/test/java/org/apache/atlas/type/TestAtlasClassificationType.java 
aaf4a6ac0a978e5eb6de41279cae1b1c82373374 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasClassificationDefStoreV1.java
 e3aa4e0b2317bec47426a914f6feae68b17851dd 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 dc6416a9b6ce7c514d3acaf32ac4cdb40dcbc030 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
 227f7cd12a9b23c3bbc1cfdc40d06616ea775ca4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
 f639ea56e6188837e069a5fcba953d9d196af0e5 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
 8638a7f17e8a17d3a6e0bfb94879b5c5406be1a3 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
 62fad5b02a7229d9bc3083690980eb063756bc29 


Diff: https://reviews.apache.org/r/61526/diff/5/

Changes: https://reviews.apache.org/r/61526/diff/4-5/


Testing
---

Testing
performed unit tests

Using postman
1) create an entityDef  aaa
2) create a classificationDef with an entitytype   - checked that it is in 
the response
3) Create a entity instance of aaa
4) add the classification to it
5) Create an entity instance with a different type 
6) Attempt to add the classification to . this fails with an informative 
message
7) Attempt to update the ClassificationDef to remove the entity type - this 
fails with an informative message
8) Attempt to update the classificationdef to add . this update works. 
9) Attempt to add an entity type that does not exist to the ClassificationDef. 
this should fail.  
10) Attempt to update an entity type that does not exist to the 
ClassificationDef. this should fail.


Thanks,

David Radley

[jira] [Commented] (ATLAS-2121) Inconsistency in basic search results due to case sensitivity of type names

[JIRA]

[ 
https://issues.apache.org/jira/browse/ATLAS-2121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164593#comment-16164593
 ] 

Péter Gergő Barna commented on ATLAS-2121:
--

this may be related to https://issues.apache.org/jira/browse/ATLAS-1925

> Inconsistency in basic search results due to case sensitivity of type names
> ---
>
> Key: ATLAS-2121
> URL: https://issues.apache.org/jira/browse/ATLAS-2121
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating, 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Péter Gergő Barna
>Priority: Critical
>
> Type names are case sensitive hence 2 types can be created with different 
> cases. 
> Example :
> 1. type1 with attributes a , b
> 2. TYPE1 with attributes c , d
> The types creation succeeds and 
> {code}
> /api/atlas/v2/types/entitydef/name/type1
> /api/atlas/v2/types/entitydef/name/TYPE1
> {code}
> return the corresponding type definitions correctly.
> When an entity e1 of type1 is created, 
> and when a BASIC search is fired with
>  typeName= type1 , e1 is returned and
>  typeName=TYPE1 also , e1 is returned.
> But in the case of DSL search , 
> typeName = type1 returns e1
> typeName = TYPE1 doesn't return any result - which is the correct behavior.
> This issue is major from tag side too because , UI Tag search is Basic and 
> returns entity tagged with tag1 when searched for tag1 or TAG1.
> Whereas DSL search with
> classification=tag1 returns entity
> classification=TAG1 does not return any entity. - which is the correct 
> behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[David Radley]

> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > 
> >
> > can we include descriptions for existing attributes and 
> > relationshipEndDefs for out of the box models as part of this JIRA?
> 
> David Radley wrote:
> I think it is better to separate the putting values into the models into 
> separate Jiras. We can then use those Jiras to review the description content 
> rather than the mechanism to introduce description.
> 
> Madhan Neethiraj wrote:
> I agree. However, I would suggest adding 'Map aliases' 
> field to struct-def and attribute-def - to enable alternate names for these 
> to be stored along with entity-def, classification-def and attribute-def. 
> This should be a trivial update and can be taken up along with addition of 
> 'description' field.
> 
> David Radley wrote:
> I am unsure what the keys and values might be for the aliases map you are 
> proposing. Are you thinking that the aliases would be instance aliases or 
> type aliases? 
> 
> We are thinking of adding external type identifications 
> (ExternalTypeIdDefs) as a new array associated with attributes, structures 
> (and the otehr defs). We are thinking the elements would be specified in the 
> typeDefs in the form:
> 
> ExternalTypeIdDefs [
> {   "scheme":"O-DEF",
> "value":"123-456",
> "description":"Relational Database",
> "ResponsibleOrg":"Open Group",
> "ReferenceDocumentationURL":"http://xxx.yyy.zzz;
> },
> {
> ...
> }
> ]
> 
> Scheme and value would be manditory.
> 
> One way to handle type aliases would be using the above mechanism: 
> {   "scheme":"alias",
> "value":"RDB",
> "description":"short hand relational database key for consumer A",
> "ResponsibleOrg":"Consumer A organisation",
> "ReferenceDocumentationURL":"http://aaa.bbb.bbb;
> },
> 
> or
> 
> {   "scheme":"alias",
> "value":"RDB"
> },
> 
> 
> This would allow these external type identifier metadata values to be 
> visible in the json models.
> 
> I was thinking of doing this piece separately in another Jira. The 
> addition of the code as-is in the Jira will in enable me to add desciptions 
> to the attributes / relationship ends in the new models. For the existing 
> Hadoop models - it might be worth having someone more expert in Hadoop 
> filling in those descriptions.
> 
> David Radley wrote:
> One other thought - if the aliases map are for instances - I wonder if 
> they could be indexed. I assume they could be specified in a search query.
> 
> Madhan Neethiraj wrote:
> We discussed about an entity-type or an attribute in Atlas being referred 
> with a different name in other meta-data standards. The 'alias' field I 
> suggested above is to capture these names along with entity-def/attribute-def 
> in Atlas. I assume it will be enough to capure only the name used in other 
> standards; if more richer information is available, how likely it is to be 
> able to capture all the details available in various standards? To keep it 
> simple, we can start with only capturing the names used in other meta-data 
> standards.

Hi Madhan,
Thanks for your quick reply. We did start with name and value pair, but felt 
the additional fields could give real value, for those that use them.  

To answer " how likely it is to be able to capture all the details available in 
various standards". I think that for the O-DEF content we are thinking about 
adding to the models, we could fill out all this content. If other standards do 
not want to or have this information, then they can omit the optional fields. 
The advantage of the "ResponsibleOrg" and "ReferenceDocumentationURL" is that 
they leave no doubt as to the author of these names, which is great for 
auditing and maybe even lineage visualizations. Just calling this alias may 
imply this is a display name - but for the use case I was thinking of, it is 
actually an external identifier in another metadata standard.  

I hope this approach keeps it simple for people wanting to add name value 
pairs, also allowing us to have the values in the model files (a very easy 
proof point to show we can work with O-DEF) and also gives us the ability to 
add the extra provenance information. Lets discuss further on another Jira if 
we need to.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185223
---


On Sept. 12, 2017, 5:13 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. 

Build failed in Jenkins: Atlas-0.8-UnitTests #53

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2120: Import API updated to detect attribute-type change

--
[...truncated 450.70 KB...]
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-webapp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
atlas-webapp ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 50 source files to 

[WARNING] 
:
 Some input files use or override a deprecated API.
[WARNING] 
:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-war-plugin:2.6:war (default-war) @ atlas-webapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [atlas-webapp] in 
[
[INFO] Processing overlay [ id org.apache.atlas:atlas-dashboardv2]
[INFO] Processing war project
[INFO] Copying webapp resources 
[
[INFO] Building jar: 

[INFO] Webapp assembled in [3204 msecs]
[INFO] Building war: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
atlas-webapp ---
[INFO] 
[INFO] --- enunciate-maven-plugin:2.8.0:docs (default) @ atlas-webapp ---
[INFO] [ENUNCIATE] Skipping enunciate per configuration.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-webapp ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[WARNING] file.encoding cannot be set as system property, use 
-Dfile.encoding=... instead

---
 T E S T S
---
Running org.apache.atlas.util.RestUtilsTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.385 sec - in 
org.apache.atlas.util.RestUtilsTest
Running org.apache.atlas.web.setup.SetupStepsTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.578 sec - in 
org.apache.atlas.web.setup.SetupStepsTest
Running org.apache.atlas.web.security.NegativeSSLAndKerberosTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.369 sec - in 
org.apache.atlas.web.security.NegativeSSLAndKerberosTest
Running org.apache.atlas.web.security.FileAuthenticationTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.122 sec - in 
org.apache.atlas.web.security.FileAuthenticationTest
Running org.apache.atlas.web.security.SSLAndKerberosTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 78.898 sec - in 
org.apache.atlas.web.security.SSLAndKerberosTest
Running org.apache.atlas.web.security.UserDaoTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.739 sec - in 
org.apache.atlas.web.security.UserDaoTest
Running org.apache.atlas.web.security.SSLTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.167 sec - in 
org.apache.atlas.web.security.SSLTest
Running org.apache.atlas.web.util.ServletsTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.804 sec - in 
org.apache.atlas.web.util.ServletsTest
Running org.apache.atlas.web.resources.AdminResourceTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.369 sec - in 
org.apache.atlas.web.resources.AdminResourceTest
Running org.apache.atlas.web.resources.TaxonomyServiceTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time 

Build failed in Jenkins: Atlas-0.8-IntegrationTests #54

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2120: Import API updated to detect attribute-type change

--
[...truncated 29.88 MB...]
2017-09-13 10:03:45,463 DEBUG - [NotificationHookConsumer thread-0:] ~ Adding 
primitive {name=description, dataType={name=string, description=null}, 
multiplicity={lower=0, upper=1, isUnique=false}, isComposite=false, 
isUnique=false, isIndexable=false, reverseAttributeName=null} from vertex 
Titan0Vertex [id=2654720] (GraphToTypedInstanceMapper:387)
2017-09-13 10:03:45,464 DEBUG - [NotificationHookConsumer thread-0:] ~ Reading 
property Asset.description from vertex[id=2654720 type=hive_process 
guid=544ea3f2-0e77-4e9b-83a2-ea902329dc81] (GraphHelper:433)
2017-09-13 10:03:45,464 DEBUG - [NotificationHookConsumer thread-0:] ~ Mapping 
attributeInfo owner (GraphToTypedInstanceMapper:144)
2017-09-13 10:03:45,465 DEBUG - [NotificationHookConsumer thread-0:] ~ Adding 
primitive {name=owner, dataType={name=string, description=null}, 
multiplicity={lower=0, upper=1, isUnique=false}, isComposite=false, 
isUnique=false, isIndexable=true, reverseAttributeName=null} from vertex 
Titan0Vertex [id=2654720] (GraphToTypedInstanceMapper:387)
2017-09-13 10:03:45,466 DEBUG - [NotificationHookConsumer thread-0:] ~ Reading 
property Asset.owner from vertex[id=2654720 type=hive_process 
guid=544ea3f2-0e77-4e9b-83a2-ea902329dc81] (GraphHelper:433)
2017-09-13 10:03:45,473 DEBUG - [NotificationHookConsumer thread-0:] ~ Sending 
message for topic ATLAS_ENTITIES: 
{"version":{"version":"1.0.0"},"message":{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"544ea3f2-0e77-4e9b-83a2-ea902329dc81","version":0,"typeName":"hive_process","state":"ACTIVE"},"typeName":"hive_process","values":{"queryId":"jenkins_20170913100152_d09cb59f-a877-4ac2-9f5e-7b87efbaa86a","name":"create
  external table tableyftvpg2ray(id int, name string) comment \u0027table 
comment\u0027  partitioned by(dt string) location 
\u0027pfile:///home/jenkins/jenkins-slave/workspace/atlas-0.8-integrationtests/addons/hive-bridge/./target/sometestpath-data-h7cuwwbkpi\u0027","startTime":"2017-09-13T10:01:52.838Z","queryPlan":"Not
 
Supported","operationType":"CREATETABLE","outputs":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"3c73163e-b6bc-4a72-983e-ad7c62dc315b","version":0,"typeName":"DataSet","state":"DELETED"}],"endTime":"2017-09-13T10:01:52.973Z","recentQueries":["create
  external table tableyftvpg2ray(id int, name string) comment \u0027table 
comment\u0027  partitioned by(dt string) location 
\u0027pfile:///home/jenkins/jenkins-slave/workspace/atlas-0.8-integrationtests/addons/hive-bridge/./target/sometestpath-data-h7cuwwbkpi\u0027"],"inputs":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"8c0d079a-47d9-4a25-8a7f-cbd4aafe4b9a","version":0,"typeName":"DataSet","state":"ACTIVE"}],"qualifiedName":"default.tableyftvpg2ray@primary:1505296912000","queryText":"create
  external table tableyftvpg2ray(id int, name string) comment \u0027table 
comment\u0027  partitioned by(dt string) location 
\u0027pfile:///home/jenkins/jenkins-slave/workspace/atlas-0.8-integrationtests/addons/hive-bridge/./target/sometestpath-data-h7cuwwbkpi\u0027","clusterName":"primary","userName":"jenkins"},"traitNames":[],"traits":{},"systemAttributes":{}},"operationType":"ENTITY_UPDATE","traits":[]}}
 (KafkaNotification:218)
2017-09-13 10:03:46,088 INFO  - [ShutdownMonitor:] ~ Stopping service 
org.apache.atlas.notification.NotificationHookConsumer (Services:65)
2017-09-13 10:03:46,088 INFO  - [ShutdownMonitor:] ~ ==> stopConsumerThreads() 
(NotificationHookConsumer:181)
2017-09-13 10:03:46,088 INFO  - [ShutdownMonitor:] ~ ==> HookConsumer 
shutdown() (NotificationHookConsumer$HookConsumer:485)
2017-09-13 10:03:46,089 INFO  - [ShutdownMonitor:] ~ 
[atlas-hook-consumer-thread], Shutting down (Logging$class:68)
2017-09-13 10:03:46,654 DEBUG - [main-SendThread(localhost:19026):] ~ ==> 
InMemoryJAASConfiguration.getAppConfigurationEntry(Client) 
(InMemoryJAASConfiguration:208)
2017-09-13 10:03:46,654 DEBUG - [main-SendThread(localhost:19026):] ~ <== 
InMemoryJAASConfiguration.getAppConfigurationEntry(Client): {} 
(InMemoryJAASConfiguration:238)
2017-09-13 10:03:46,655 WARN  - [main-SendThread(localhost:19026):] ~ Session 
0x15e7aad19fb0001 for server null, unexpected error, closing socket connection 
and attempting reconnect (ClientCnxn$SendThread:1102)
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
at 
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at 

Build failed in Jenkins: Atlas-master-UnitTests #70

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2120: Import API updated to detect attribute-type change

--
[...truncated 454.08 KB...]
[WARNING] 
:
 Some input files use unchecked or unsafe operations.
[WARNING] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-war-plugin:2.6:war (default-war) @ atlas-webapp ---
[INFO] Packaging webapp
[INFO] Assembling webapp [atlas-webapp] in 
[
[INFO] Processing overlay [ id org.apache.atlas:atlas-dashboardv2]
[INFO] Processing war project
[INFO] Copying webapp resources 
[
[INFO] Building jar: 

[INFO] Webapp assembled in [835 msecs]
[INFO] Building war: 

[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
atlas-webapp ---
[INFO] 
[INFO] --- enunciate-maven-plugin:2.8.0:docs (default) @ atlas-webapp ---
[INFO] [ENUNCIATE] Skipping enunciate per configuration.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-webapp ---
[WARNING] The parameter forkMode is deprecated since version 2.14. Use 
forkCount and reuseForks instead.
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider
[WARNING] file.encoding cannot be set as system property, use 
-Dfile.encoding=... instead

---
 T E S T S
---
Running org.apache.atlas.notification.NotificationEntityChangeListenerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.307 sec - in 
org.apache.atlas.notification.NotificationEntityChangeListenerTest
Running org.apache.atlas.notification.AdaptiveWaiterTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.134 sec - in 
org.apache.atlas.notification.AdaptiveWaiterTest
Running org.apache.atlas.notification.NotificationHookConsumerKafkaTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.513 sec - in 
org.apache.atlas.notification.NotificationHookConsumerKafkaTest
Running org.apache.atlas.notification.NotificationHookConsumerTest
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.961 sec - in 
org.apache.atlas.notification.NotificationHookConsumerTest
Running org.apache.atlas.util.RestUtilsTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.439 sec - in 
org.apache.atlas.util.RestUtilsTest
Running org.apache.atlas.web.setup.SetupStepsTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.604 sec - in 
org.apache.atlas.web.setup.SetupStepsTest
Running org.apache.atlas.web.security.NegativeSSLAndKerberosTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 31.439 sec <<< 
FAILURE! - in org.apache.atlas.web.security.NegativeSSLAndKerberosTest
testUnsecuredClient(org.apache.atlas.web.security.NegativeSSLAndKerberosTest)  
Time elapsed: 0.325 sec  <<< FAILURE!
java.lang.AssertionError: expected: but was:
at org.testng.Assert.fail(Assert.java:89)
at org.testng.Assert.failNotEquals(Assert.java:489)
at org.testng.Assert.assertTrue(Assert.java:37)
at org.testng.Assert.assertTrue(Assert.java:47)
at 
org.apache.atlas.web.security.NegativeSSLAndKerberosTest.testUnsecuredClient(NegativeSSLAndKerberosTest.java:139)

Running org.apache.atlas.web.security.SSLAndKerberosTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 31.572 sec - in 
org.apache.atlas.web.security.SSLAndKerberosTest
Running org.apache.atlas.web.security.SSLTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.275 sec - in 
org.apache.atlas.web.security.SSLTest
Running org.apache.atlas.web.security.FileAuthenticationTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.451 sec - in 
org.apache.atlas.web.security.FileAuthenticationTest
Running org.apache.atlas.web.security.UserDaoTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.702 sec - in 
org.apache.atlas.web.security.UserDaoTest
Running org.apache.atlas.web.resources.AdminExportTest
Tests run: 0, Failures: 0, 

[jira] [Created] (ATLAS-2132) Error code during invalid file path/unreadable file provided during import

[Sharmadha Sainath (JIRA)]

Sharmadha Sainath created ATLAS-2132:


 Summary: Error code during invalid file path/unreadable file 
provided during import
 Key: ATLAS-2132
 URL: https://issues.apache.org/jira/browse/ATLAS-2132
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.9-incubating
Reporter: Sharmadha Sainath
Priority: Minor


When firing import command using 
{code}
/api/atlas/admin/importfile
{code}
when file provided in the import_options.json doesn't have read permission or 
file is not present  , Atlas throws 500 Internal server error but with proper 
error message. 

Example  :
{code}
{"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
 invalid parameters: /exportimport/db5.zip: file not found"}
{code}

{code}
{"errorCode":"ATLAS-500-00-001","errorMessage":"org.apache.atlas.exception.AtlasBaseException:
 java.io.IOException: File '/exportimport/db6.zip' cannot be read"}
{code}

Expected that Atlas would throw 400 Bad Request instead of 500 Internal Server 
error.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: Atlas-master-IntegrationTests #92

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2120: Import API updated to detect attribute-type change

--
[...truncated 457.10 KB...]
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "GET 
/api/atlas/v2/types/classificationdef/guid/blah HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "GET 
/api/atlas/v2/types/entitydef/name/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "GET 
/api/atlas/v2/types/entitydef/guid/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "GET 
/api/atlas/v2/types/typedefs/?supertype=ApbTmNDLFU6=CLASS HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "GET 
/api/atlas/v2/types/typedefs/?notsupertype=BGjwPvowNmq=ApbTmNDLFU6=CLASS
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:41 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "PUT /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedefs/?type=ENTITY HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 409 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET /api/atlas/types/blah HTTP/1.1" 
404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedef/name/class_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedef/name/secure_trait HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedef/name/pii_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedef/name/ssn_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedef/name/salary_trait HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "GET 
/api/atlas/v2/types/typedef/name/sox_trait HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:42 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "GET /api/atlas/types?type=TRAIT 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "GET 
/api/atlas/v2/types/typedef/name/AkF91f6eQ51 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "GET 
/api/atlas/v2/types/typedef/name/A1n4V3SiB450 HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "GET 
/api/atlas/v2/types/typedef/name/BukbxZb6os9 HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "GET 
/api/atlas/v2/types/typedef/name/CzYyY7ugheJ HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "GET 
/api/atlas/types?type=CLASS=AkF91f6eQ51=BukbxZb6os9 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:43 +] "PUT /api/atlas/types HTTP/1.1" 200 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [13/Sep/2017:09:43:44 +] "GET /api/atlas/types/PKucwOyGeC 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
Tests run: 143, Failures: 16, Errors: 0, Skipped: 100, Time elapsed: 20.496 sec 
<<< FAILURE! - in TestSuite
setUp(org.apache.atlas.web.integration.EntityDiscoveryJerseyResourceIT)  Time 
elapsed: 3.774 sec  <<< FAILURE!
org.apache.atlas.AtlasServiceException: Metadata service API 
org.apache.atlas.AtlasBaseClient$APIInfo@13863b4c failed with status 400 (Bad 
Request) 

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[Madhan Neethiraj]

> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > 
> >
> > can we include descriptions for existing attributes and 
> > relationshipEndDefs for out of the box models as part of this JIRA?
> 
> David Radley wrote:
> I think it is better to separate the putting values into the models into 
> separate Jiras. We can then use those Jiras to review the description content 
> rather than the mechanism to introduce description.
> 
> Madhan Neethiraj wrote:
> I agree. However, I would suggest adding 'Map aliases' 
> field to struct-def and attribute-def - to enable alternate names for these 
> to be stored along with entity-def, classification-def and attribute-def. 
> This should be a trivial update and can be taken up along with addition of 
> 'description' field.
> 
> David Radley wrote:
> I am unsure what the keys and values might be for the aliases map you are 
> proposing. Are you thinking that the aliases would be instance aliases or 
> type aliases? 
> 
> We are thinking of adding external type identifications 
> (ExternalTypeIdDefs) as a new array associated with attributes, structures 
> (and the otehr defs). We are thinking the elements would be specified in the 
> typeDefs in the form:
> 
> ExternalTypeIdDefs [
> {   "scheme":"O-DEF",
> "value":"123-456",
> "description":"Relational Database",
> "ResponsibleOrg":"Open Group",
> "ReferenceDocumentationURL":"http://xxx.yyy.zzz;
> },
> {
> ...
> }
> ]
> 
> Scheme and value would be manditory.
> 
> One way to handle type aliases would be using the above mechanism: 
> {   "scheme":"alias",
> "value":"RDB",
> "description":"short hand relational database key for consumer A",
> "ResponsibleOrg":"Consumer A organisation",
> "ReferenceDocumentationURL":"http://aaa.bbb.bbb;
> },
> 
> or
> 
> {   "scheme":"alias",
> "value":"RDB"
> },
> 
> 
> This would allow these external type identifier metadata values to be 
> visible in the json models.
> 
> I was thinking of doing this piece separately in another Jira. The 
> addition of the code as-is in the Jira will in enable me to add desciptions 
> to the attributes / relationship ends in the new models. For the existing 
> Hadoop models - it might be worth having someone more expert in Hadoop 
> filling in those descriptions.
> 
> David Radley wrote:
> One other thought - if the aliases map are for instances - I wonder if 
> they could be indexed. I assume they could be specified in a search query.

We discussed about an entity-type or an attribute in Atlas being referred with 
a different name in other meta-data standards. The 'alias' field I suggested 
above is to capture these names along with entity-def/attribute-def in Atlas. I 
assume it will be enough to capure only the name used in other standards; if 
more richer information is available, how likely it is to be able to capture 
all the details available in various standards? To keep it simple, we can start 
with only capturing the names used in other meta-data standards.


- Madhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185223
---


On Sept. 12, 2017, 5:13 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62244/
> ---
> 
> (Updated Sept. 12, 2017, 5:13 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS2058 Add description to attributedefs and relationship enddefs
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java
>  01e5ce738e8a5310cee928560364d4a38f5c6e18 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> 3a5c43ac7e6d0a7f683c24d13d6c841de4d66a14 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  c3c42be86d8c9fd282fdc1428c8fd78e338a3cfb 
> 
> 
> Diff: https://reviews.apache.org/r/62244/diff/1/
> 
> 
> Testing
> ---
> 
> Create an entity attribute, relationship attribute and a relationship end 
> with a description and then issue getTypeDefs to check they are there.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[David Radley]

> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > 
> >
> > can we include descriptions for existing attributes and 
> > relationshipEndDefs for out of the box models as part of this JIRA?
> 
> David Radley wrote:
> I think it is better to separate the putting values into the models into 
> separate Jiras. We can then use those Jiras to review the description content 
> rather than the mechanism to introduce description.
> 
> Madhan Neethiraj wrote:
> I agree. However, I would suggest adding 'Map aliases' 
> field to struct-def and attribute-def - to enable alternate names for these 
> to be stored along with entity-def, classification-def and attribute-def. 
> This should be a trivial update and can be taken up along with addition of 
> 'description' field.
> 
> David Radley wrote:
> I am unsure what the keys and values might be for the aliases map you are 
> proposing. Are you thinking that the aliases would be instance aliases or 
> type aliases? 
> 
> We are thinking of adding external type identifications 
> (ExternalTypeIdDefs) as a new array associated with attributes, structures 
> (and the otehr defs). We are thinking the elements would be specified in the 
> typeDefs in the form:
> 
> ExternalTypeIdDefs [
> {   "scheme":"O-DEF",
> "value":"123-456",
> "description":"Relational Database",
> "ResponsibleOrg":"Open Group",
> "ReferenceDocumentationURL":"http://xxx.yyy.zzz;
> },
> {
> ...
> }
> ]
> 
> Scheme and value would be manditory.
> 
> One way to handle type aliases would be using the above mechanism: 
> {   "scheme":"alias",
> "value":"RDB",
> "description":"short hand relational database key for consumer A",
> "ResponsibleOrg":"Consumer A organisation",
> "ReferenceDocumentationURL":"http://aaa.bbb.bbb;
> },
> 
> or
> 
> {   "scheme":"alias",
> "value":"RDB"
> },
> 
> 
> This would allow these external type identifier metadata values to be 
> visible in the json models.
> 
> I was thinking of doing this piece separately in another Jira. The 
> addition of the code as-is in the Jira will in enable me to add desciptions 
> to the attributes / relationship ends in the new models. For the existing 
> Hadoop models - it might be worth having someone more expert in Hadoop 
> filling in those descriptions.

One other thought - if the aliases map are for instances - I wonder if they 
could be indexed. I assume they could be specified in a search query.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185223
---


On Sept. 12, 2017, 5:13 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62244/
> ---
> 
> (Updated Sept. 12, 2017, 5:13 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS2058 Add description to attributedefs and relationship enddefs
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java
>  01e5ce738e8a5310cee928560364d4a38f5c6e18 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> 3a5c43ac7e6d0a7f683c24d13d6c841de4d66a14 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  c3c42be86d8c9fd282fdc1428c8fd78e338a3cfb 
> 
> 
> Diff: https://reviews.apache.org/r/62244/diff/1/
> 
> 
> Testing
> ---
> 
> Create an entity attribute, relationship attribute and a relationship end 
> with a description and then issue getTypeDefs to check they are there.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[David Radley]

> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > 
> >
> > can we include descriptions for existing attributes and 
> > relationshipEndDefs for out of the box models as part of this JIRA?
> 
> David Radley wrote:
> I think it is better to separate the putting values into the models into 
> separate Jiras. We can then use those Jiras to review the description content 
> rather than the mechanism to introduce description.
> 
> Madhan Neethiraj wrote:
> I agree. However, I would suggest adding 'Map aliases' 
> field to struct-def and attribute-def - to enable alternate names for these 
> to be stored along with entity-def, classification-def and attribute-def. 
> This should be a trivial update and can be taken up along with addition of 
> 'description' field.

I am unsure what the keys and values might be for the aliases map you are 
proposing. Are you thinking that the aliases would be instance aliases or type 
aliases? 

We are thinking of adding external type identifications (ExternalTypeIdDefs) as 
a new array associated with attributes, structures (and the otehr defs). We are 
thinking the elements would be specified in the typeDefs in the form:

ExternalTypeIdDefs [
{   "scheme":"O-DEF",
"value":"123-456",
"description":"Relational Database",
"ResponsibleOrg":"Open Group",
"ReferenceDocumentationURL":"http://xxx.yyy.zzz;
},
{
...
}
]

Scheme and value would be manditory.

One way to handle type aliases would be using the above mechanism: 
{   "scheme":"alias",
"value":"RDB",
"description":"short hand relational database key for consumer A",
"ResponsibleOrg":"Consumer A organisation",
"ReferenceDocumentationURL":"http://aaa.bbb.bbb;
},

or

{   "scheme":"alias",
"value":"RDB"
},


This would allow these external type identifier metadata values to be visible 
in the json models.

I was thinking of doing this piece separately in another Jira. The addition of 
the code as-is in the Jira will in enable me to add desciptions to the 
attributes / relationship ends in the new models. For the existing Hadoop 
models - it might be worth having someone more expert in Hadoop filling in 
those descriptions.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185223
---


On Sept. 12, 2017, 5:13 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62244/
> ---
> 
> (Updated Sept. 12, 2017, 5:13 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS2058 Add description to attributedefs and relationship enddefs
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java
>  01e5ce738e8a5310cee928560364d4a38f5c6e18 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> 3a5c43ac7e6d0a7f683c24d13d6c841de4d66a14 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  c3c42be86d8c9fd282fdc1428c8fd78e338a3cfb 
> 
> 
> Diff: https://reviews.apache.org/r/62244/diff/1/
> 
> 
> Testing
> ---
> 
> Create an entity attribute, relationship attribute and a relationship end 
> with a description and then issue getTypeDefs to check they are there.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 62227: ATLAS-2120: Import API: Type Attribute Type Definition Matched

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62227/#review185270
---




repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java
Lines 121 (patched)


I would suggest to define an error code for this case, instead of using 
INTERNAL_ERROR.


- Madhan Neethiraj


On Sept. 11, 2017, 9:03 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62227/
> ---
> 
> (Updated Sept. 11, 2017, 9:03 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sharmadha Sainath.
> 
> 
> Bugs: ATLAS-2120
> https://issues.apache.org/jira/browse/ATLAS-2120
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is a minor change, that performs additional check during import process. 
> 
> Import process now matches type definition for the types being imported. It 
> fails and logs exception if types do not match.
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/Import-API-Options.twiki cc09fc7 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java
>  68bc583 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  42f7011 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
>  d80fab0 
>   repository/src/test/resources/hdfs_path1.zip PRE-CREATION 
>   repository/src/test/resources/tag1.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/62227/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> - Additional failure unit tests added.
> 
> **Functional tests**
> - Executed success and failure scenarios.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

[jira] [Commented] (ATLAS-2131) Issues during navigation from table in Tables tab with Include historical entities checked

[Sharmadha Sainath (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16164207#comment-16164207
 ] 

Sharmadha Sainath commented on ATLAS-2131:
--

CC : [~kevalbhatt] [~madhan.neethiraj]

> Issues during navigation from table in Tables tab with Include historical 
> entities checked
> --
>
> Key: ATLAS-2131
> URL: https://issues.apache.org/jira/browse/ATLAS-2131
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
> Attachments: TablesInDbNavigation.mov
>
>
> Following is one of the scenarios where there is issue when navigating. Not 
> sure what other scenarios can also lead to the issue.
> 1.Created 100 hive_table entities in a database db1.
> 2. Fired basic query hive_db = db1.
> 3. In the Tables tab of db1 , all 100 tables were listed.
> 4.Checked Include historical entities check box. Now , URL changed to
> {code}
> localhost:21000/#!/search/searchResult?guid=8c7549db-c514-419e-bc2c-f3fe9e3c34f6=relationship=true=true
> {code} 
> 5.Clicked on any of the tables in the result which landed in the detailsPage 
> of the table.
> 6.Now , when "Back To Results" button is clicked , URL changed to 
> {code}
> http://localhost:21000/#!/search/searchResult?guid=8c7549db-c514-419e-bc2c-f3fe9e3c34f6=relationship=true=true
> {code}
>  and landed in Atlas home page (ie. Search page) with all search cleared.
> 7. Fired some basic query and clicked Search . Search was not functioning.
> 8.Switched from Basic -  DSL - Basic . Now fired search again . This time 
> search was working . But  Include Historical entities was checked on by 
> default.
> 9. If Include historical entities was not checked in Tables tab and any 
> entity was selected , this issue was not seen.
> Added a screen recording of the scenario explained above.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2131) Issues during navigation from table in Tables tab with Include historical entities checked

[Sharmadha Sainath (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sharmadha Sainath updated ATLAS-2131:
-
Summary: Issues during navigation from table in Tables tab with Include 
historical entities checked  (was: Issues during navigation from deleted table 
in Tables tab)

> Issues during navigation from table in Tables tab with Include historical 
> entities checked
> --
>
> Key: ATLAS-2131
> URL: https://issues.apache.org/jira/browse/ATLAS-2131
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
> Attachments: TablesInDbNavigation.mov
>
>
> Following is one of the scenarios where there is issue when navigating. Not 
> sure what other scenarios can also lead to the issue.
> 1.Created 100 hive_table entities in a database db1.
> 2. Fired basic query hive_db = db1.
> 3. In the Tables tab of db1 , all 100 tables were listed.
> 4.Checked Include historical entities check box. Now , URL changed to
> {code}
> localhost:21000/#!/search/searchResult?guid=8c7549db-c514-419e-bc2c-f3fe9e3c34f6=relationship=true=true
> {code} 
> 5.Clicked on any of the tables in the result which landed in the detailsPage 
> of the table.
> 6.Now , when "Back To Results" button is clicked , URL changed to 
> {code}
> http://localhost:21000/#!/search/searchResult?guid=8c7549db-c514-419e-bc2c-f3fe9e3c34f6=relationship=true=true
> {code}
>  and landed in Atlas home page (ie. Search page) with all search cleared.
> 7. Fired some basic query and clicked Search . Search was not functioning.
> 8.Switched from Basic -  DSL - Basic . Now fired search again . This time 
> search was working . But  Include Historical entities was checked on by 
> default.
> 9. If Include historical entities was not checked in Tables tab and any 
> entity was selected , this issue was not seen.
> Added a screen recording of the scenario explained above.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2131) Issues during navigation from deleted table in Tables tab

[Sharmadha Sainath (JIRA)]

Sharmadha Sainath created ATLAS-2131:


 Summary: Issues during navigation from deleted table in Tables tab
 Key: ATLAS-2131
 URL: https://issues.apache.org/jira/browse/ATLAS-2131
 Project: Atlas
  Issue Type: Bug
  Components: atlas-webui
Affects Versions: 0.9-incubating
Reporter: Sharmadha Sainath
 Attachments: TablesInDbNavigation.mov

Following is one of the scenarios where there is issue when navigating. Not 
sure what other scenarios can also lead to the issue.

1.Created 100 hive_table entities in a database db1.
2. Fired basic query hive_db = db1.
3. In the Tables tab of db1 , all 100 tables were listed.
4.Checked Include historical entities check box. Now , URL changed to
{code}
localhost:21000/#!/search/searchResult?guid=8c7549db-c514-419e-bc2c-f3fe9e3c34f6=relationship=true=true
{code} 
5.Clicked on any of the tables in the result which landed in the detailsPage of 
the table.
6.Now , when "Back To Results" button is clicked , URL changed to 
{code}
http://localhost:21000/#!/search/searchResult?guid=8c7549db-c514-419e-bc2c-f3fe9e3c34f6=relationship=true=true
{code}
 and landed in Atlas home page (ie. Search page) with all search cleared.
7. Fired some basic query and clicked Search . Search was not functioning.
8.Switched from Basic -  DSL - Basic . Now fired search again . This time 
search was working . But  Include Historical entities was checked on by default.
9. If Include historical entities was not checked in Tables tab and any entity 
was selected , this issue was not seen.

Added a screen recording of the scenario explained above.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)