[jira] [Commented] (ATLAS-633) "kafka.common.NoReplicaOnlineException" exception on atlas startup

2017-08-15 Thread Ayub Pathan (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128277#comment-16128277
 ] 

Ayub Pathan commented on ATLAS-633:
---

[~bill] This issue is seen in embedded mode of Atlas and I believe this is a 
limitation from ATlas-Kafka integration. Since I am using non-embedded mode of 
Atlas, I didn't pursue on this route.

> "kafka.common.NoReplicaOnlineException" exception on atlas startup
> --
>
> Key: ATLAS-633
> URL: https://issues.apache.org/jira/browse/ATLAS-633
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ayub Pathan
>
> "kafka.common.NoReplicaOnlineException" exception on atlas startup
> {noformat}
> 2016-04-05 11:01:54,811 INFO  - [main:] ~ [Socket Server on Broker 1], 
> Started 1 acceptor threads (SocketServer:68)
> 2016-04-05 11:01:54,828 INFO  - [ExpirationReaper-1:] ~ [ExpirationReaper-1], 
> Starting  (DelayedOperationPurgatory$ExpiredOperationReaper:68)
> 2016-04-05 11:01:54,828 INFO  - [ExpirationReaper-1:] ~ [ExpirationReaper-1], 
> Starting  (DelayedOperationPurgatory$ExpiredOperationReaper:68)
> 2016-04-05 11:01:54,852 INFO  - [main:] ~ [Controller 1]: Controller starting 
> up (KafkaController:68)
> 2016-04-05 11:01:54,862 INFO  - [main:] ~ Creating /controller (is it secure? 
> false) (ZKCheckedEphemeral:68)
> 2016-04-05 11:01:54,867 INFO  - [main:] ~ Result of znode creation is: OK 
> (ZKCheckedEphemeral:68)
> 2016-04-05 11:01:54,868 INFO  - [main:] ~ 1 successfully elected as leader 
> (ZookeeperLeaderElector:68)
> 2016-04-05 11:01:54,868 INFO  - [main:] ~ [Controller 1]: Broker 1 starting 
> become controller state transition (KafkaController:68)
> 2016-04-05 11:01:54,870 INFO  - [main:] ~ [Controller 1]: Initialized 
> controller epoch to 1 and zk version 0 (KafkaController:68)
> 2016-04-05 11:01:54,872 INFO  - [main:] ~ [Controller 1]: Controller 1 
> incremented epoch to 2 (KafkaController:68)
> 2016-04-05 11:01:54,939 INFO  - [main:] ~ [Controller 1]: Partitions 
> undergoing preferred replica election:  (KafkaController:68)
> 2016-04-05 11:01:54,939 INFO  - [main:] ~ [Controller 1]: Partitions that 
> completed preferred replica election:  (KafkaController:68)
> 2016-04-05 11:01:54,940 INFO  - [main:] ~ [Controller 1]: Resuming preferred 
> replica election for partitions:  (KafkaController:68)
> 2016-04-05 11:01:54,943 INFO  - [main:] ~ [Controller 1]: Partitions being 
> reassigned: Map() (KafkaController:68)
> 2016-04-05 11:01:54,944 INFO  - [main:] ~ [Controller 1]: Partitions already 
> reassigned: List() (KafkaController:68)
> 2016-04-05 11:01:54,945 INFO  - [main:] ~ [Controller 1]: Resuming 
> reassignment of partitions: Map() (KafkaController:68)
> 2016-04-05 11:01:54,949 INFO  - [main:] ~ [Controller 1]: List of topics to 
> be deleted:  (KafkaController:68)
> 2016-04-05 11:01:54,949 INFO  - [main:] ~ [Controller 1]: List of topics 
> ineligible for deletion: ATLAS_HOOK,ATLAS_ENTITIES (KafkaController:68)
> 2016-04-05 11:01:54,951 INFO  - [main:] ~ [Controller 1]: Currently active 
> brokers in the cluster: Set() (KafkaController:68)
> 2016-04-05 11:01:54,952 INFO  - [main:] ~ [Controller 1]: Currently shutting 
> brokers in the cluster: Set() (KafkaController:68)
> 2016-04-05 11:01:54,952 INFO  - [main:] ~ [Controller 1]: Current list of 
> topics in the cluster: Set(ATLAS_ENTITIES, ATLAS_HOOK) (KafkaController:68)
> 2016-04-05 11:01:54,956 INFO  - [main:] ~ [Replica state machine on 
> controller 1]: Started replica state machine with initial state -> 
> Map([Topic=ATLAS_ENTITIES,Partition=0,Replica=1] -> 
> ReplicaDeletionIneligible, [Topic=ATLAS_HOOK,Partition=0,Replica=1] -> 
> ReplicaDeletionIneligible) (ReplicaStateMachine:68)
> 2016-04-05 11:01:54,977 ERROR - [main:] ~ Controller 1 epoch 2 initiated 
> state change for partition [ATLAS_HOOK,0] from OfflinePartition to 
> OnlinePartition failed (logger:103)
> kafka.common.NoReplicaOnlineException: No replica for partition 
> [ATLAS_HOOK,0] is alive. Live brokers are: [Set()], Assigned replicas are: 
> [List(1)]
>   at 
> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:75)
>   at 
> kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:345)
>   at 
> kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:205)
>   at 
> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:120)
>   at 
> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:117)
>   at 
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scal

[jira] [Commented] (ATLAS-633) "kafka.common.NoReplicaOnlineException" exception on atlas startup

2017-08-15 Thread Bill (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128217#comment-16128217
 ] 

Bill commented on ATLAS-633:


[~ayubkhan]  Hi, is this problem you finally solved? :D

> "kafka.common.NoReplicaOnlineException" exception on atlas startup
> --
>
> Key: ATLAS-633
> URL: https://issues.apache.org/jira/browse/ATLAS-633
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Ayub Pathan
>
> "kafka.common.NoReplicaOnlineException" exception on atlas startup
> {noformat}
> 2016-04-05 11:01:54,811 INFO  - [main:] ~ [Socket Server on Broker 1], 
> Started 1 acceptor threads (SocketServer:68)
> 2016-04-05 11:01:54,828 INFO  - [ExpirationReaper-1:] ~ [ExpirationReaper-1], 
> Starting  (DelayedOperationPurgatory$ExpiredOperationReaper:68)
> 2016-04-05 11:01:54,828 INFO  - [ExpirationReaper-1:] ~ [ExpirationReaper-1], 
> Starting  (DelayedOperationPurgatory$ExpiredOperationReaper:68)
> 2016-04-05 11:01:54,852 INFO  - [main:] ~ [Controller 1]: Controller starting 
> up (KafkaController:68)
> 2016-04-05 11:01:54,862 INFO  - [main:] ~ Creating /controller (is it secure? 
> false) (ZKCheckedEphemeral:68)
> 2016-04-05 11:01:54,867 INFO  - [main:] ~ Result of znode creation is: OK 
> (ZKCheckedEphemeral:68)
> 2016-04-05 11:01:54,868 INFO  - [main:] ~ 1 successfully elected as leader 
> (ZookeeperLeaderElector:68)
> 2016-04-05 11:01:54,868 INFO  - [main:] ~ [Controller 1]: Broker 1 starting 
> become controller state transition (KafkaController:68)
> 2016-04-05 11:01:54,870 INFO  - [main:] ~ [Controller 1]: Initialized 
> controller epoch to 1 and zk version 0 (KafkaController:68)
> 2016-04-05 11:01:54,872 INFO  - [main:] ~ [Controller 1]: Controller 1 
> incremented epoch to 2 (KafkaController:68)
> 2016-04-05 11:01:54,939 INFO  - [main:] ~ [Controller 1]: Partitions 
> undergoing preferred replica election:  (KafkaController:68)
> 2016-04-05 11:01:54,939 INFO  - [main:] ~ [Controller 1]: Partitions that 
> completed preferred replica election:  (KafkaController:68)
> 2016-04-05 11:01:54,940 INFO  - [main:] ~ [Controller 1]: Resuming preferred 
> replica election for partitions:  (KafkaController:68)
> 2016-04-05 11:01:54,943 INFO  - [main:] ~ [Controller 1]: Partitions being 
> reassigned: Map() (KafkaController:68)
> 2016-04-05 11:01:54,944 INFO  - [main:] ~ [Controller 1]: Partitions already 
> reassigned: List() (KafkaController:68)
> 2016-04-05 11:01:54,945 INFO  - [main:] ~ [Controller 1]: Resuming 
> reassignment of partitions: Map() (KafkaController:68)
> 2016-04-05 11:01:54,949 INFO  - [main:] ~ [Controller 1]: List of topics to 
> be deleted:  (KafkaController:68)
> 2016-04-05 11:01:54,949 INFO  - [main:] ~ [Controller 1]: List of topics 
> ineligible for deletion: ATLAS_HOOK,ATLAS_ENTITIES (KafkaController:68)
> 2016-04-05 11:01:54,951 INFO  - [main:] ~ [Controller 1]: Currently active 
> brokers in the cluster: Set() (KafkaController:68)
> 2016-04-05 11:01:54,952 INFO  - [main:] ~ [Controller 1]: Currently shutting 
> brokers in the cluster: Set() (KafkaController:68)
> 2016-04-05 11:01:54,952 INFO  - [main:] ~ [Controller 1]: Current list of 
> topics in the cluster: Set(ATLAS_ENTITIES, ATLAS_HOOK) (KafkaController:68)
> 2016-04-05 11:01:54,956 INFO  - [main:] ~ [Replica state machine on 
> controller 1]: Started replica state machine with initial state -> 
> Map([Topic=ATLAS_ENTITIES,Partition=0,Replica=1] -> 
> ReplicaDeletionIneligible, [Topic=ATLAS_HOOK,Partition=0,Replica=1] -> 
> ReplicaDeletionIneligible) (ReplicaStateMachine:68)
> 2016-04-05 11:01:54,977 ERROR - [main:] ~ Controller 1 epoch 2 initiated 
> state change for partition [ATLAS_HOOK,0] from OfflinePartition to 
> OnlinePartition failed (logger:103)
> kafka.common.NoReplicaOnlineException: No replica for partition 
> [ATLAS_HOOK,0] is alive. Live brokers are: [Set()], Assigned replicas are: 
> [List(1)]
>   at 
> kafka.controller.OfflinePartitionLeaderSelector.selectLeader(PartitionLeaderSelector.scala:75)
>   at 
> kafka.controller.PartitionStateMachine.electLeaderForPartition(PartitionStateMachine.scala:345)
>   at 
> kafka.controller.PartitionStateMachine.kafka$controller$PartitionStateMachine$$handleStateChange(PartitionStateMachine.scala:205)
>   at 
> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:120)
>   at 
> kafka.controller.PartitionStateMachine$$anonfun$triggerOnlinePartitionStateChange$3.apply(PartitionStateMachine.scala:117)
>   at 
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
>   at 
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
>   at 
> scala.collection.mutable.HashMap$$anonfun$f

[jira] [Updated] (ATLAS-1218) Atlas says it is started but does not accept REST requests

2017-08-15 Thread Richard Ding (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated ATLAS-1218:

Attachment: ATLAS-1218_1.patch

The patch is simplified using Python socket module.

The patch was tested with commands:
{code}
$ bin/atlas-start.py
{code}
and 
{code}
$ bin/atlas-start.py -port 31000
{code}
 Also tested with TLS enabled.

> Atlas says it is started but does not accept REST requests
> --
>
> Key: ATLAS-1218
> URL: https://issues.apache.org/jira/browse/ATLAS-1218
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: David Radley
>Assignee: Richard Ding
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1218_1.patch, ATLAS-1218.patch
>
>
> I start Atlas on the command line and it tells me is started - but REST 
> requests are not accepted immediately. There appears to be asynchronous 
> background processes that have not yet initialized. I feel that when Atlas 
> says it is started it should be open for business and accept REST requests.  
> a few seconds later REST requests are accepted. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61561: ATLAS-1218: Atlas says it is started but does not accept REST requests

2017-08-15 Thread Richard Ding via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61561/
---

(Updated Aug. 15, 2017, 10:08 p.m.)


Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
Subramanian.


Changes
---

The new patch simplified the code with python socket module.


Bugs: ATLAS-1218
https://issues.apache.org/jira/browse/ATLAS-1218


Repository: atlas


Description
---

This patch implements a wait_for_startup function modeled after 
wait_for_shutdown function.


Diffs (updated)
-

  distro/src/bin/atlas_config.py 1be9ca2b 
  distro/src/bin/atlas_start.py a6a3455c 


Diff: https://reviews.apache.org/r/61561/diff/3/

Changes: https://reviews.apache.org/r/61561/diff/2-3/


Testing
---

Here are the output of atlas_start and atlas_stop script:
```
starting atlas on port 21000

Apache Atlas Server started!!!
```
and
```
stopping atlas
Apache Atlas Server stopped!!!
```


Thanks,

Richard Ding

[jira] [Updated] (ATLAS-2012) Docker - image & hub - for Atlas

2017-08-15 Thread Richard Ding (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated ATLAS-2012:

Attachment: atlas_docker_2.patch

I tested the _Dockerfile_ on Mac and the Docker image was built successfully.

The attached patch just removes the references to hbase in the comments and 
also the Windows newlines.

I think it is ready for the review board.

> Docker - image & hub - for Atlas
> 
>
> Key: ATLAS-2012
> URL: https://issues.apache.org/jira/browse/ATLAS-2012
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nigel Jones
>Assignee: Péter Gergő Barna
> Attachments: atlas_docker1.patch, atlas_docker_2.patch, 
> atlas_docker.patch
>
>
> Docker is increasingly become a standard way of easily running components in 
> a flexible manner, whether for development, production, or test
> I feel there are a few things we can do with docker that will aid Atlas's 
> appeal
> 1. We could provide a simple example of how to create a docker image from the 
> Atlas build. This could be published on the wiki & it would make it easier 
> for developers to use Atlas within a docker environment - mostly by 
> addressing how to configure & start up
> 2. We could automatically generate a docker image as part of the build 
> process. This builds on #1 by automating the creation of the image & making 
> it "just part of the build"
> 3. We could publish the docker image from #2, for releases, to dockerhub, 
> making it near trivial for any developer to easily pull down and experiment 
> with Atlas. Full source of course would be provided/documented so that a user 
> could customize as needed for their environment plus of course improve what 
> is provided with the core project
> 4. We could work with other teams especially ranger, to allow similar efforts 
> in other teams to easily work together & be orchestrated 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 61670: ATLAS-1880: UTs

2017-08-15 Thread Apoorv Naik 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61670/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-1954
https://issues.apache.org/jira/browse/ATLAS-1954


Repository: atlas


Description
---

UTs for ATLAS-1880

TODO: Add complex nesting cases


Diffs
-

  
repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
 82692cfb 
  
repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
 5a0739ce 
  repository/src/test/java/org/apache/atlas/utils/TestResourceFileUtils.java 
PRE-CREATION 
  repository/src/test/resources/DefaultDB-3-New.zip PRE-CREATION 
  repository/src/test/resources/hive-db-50-tables.zip PRE-CREATION 
  
repository/src/test/resources/json/search-parameters/hive_column-contains.json 
PRE-CREATION 
  
repository/src/test/resources/json/search-parameters/hive_column-endsWith.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-contains.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-endsWith.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-eq.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-gt.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-gte.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-in.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-like.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-lt.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-lte.json 
PRE-CREATION 
  repository/src/test/resources/json/search-parameters/hive_table-neq.json 
PRE-CREATION 
  
repository/src/test/resources/json/search-parameters/hive_table-startsWith.json 
PRE-CREATION 


Diff: https://reviews.apache.org/r/61670/diff/1/


Testing
---

mvn clean package executes successfully


Thanks,

Apoorv Naik

Re: [ANNOUNCE] New Committer: David Radley

2017-08-15 Thread Apoorv Naik 
Congratulations David.





On 8/15/17, 10:20 AM, "Ayub Khan Pathan"  wrote:

>Congratulations, David.
>
>On 8/13/17, 11:07 PM, "Madhan Neethiraj"  wrote:
>
>Atlas community,
>
> 
>
>Apache Atlas PMC has asked David Radley to become a committer and we are 
> pleased to announce that he has accepted.
>
> 
>
>Welcome David, great to have you on board.
>
> 
>
>On behalf of Apache Atlas PMC,
>
> 
>
>Madhan
>
> 
>
> 
>
>
>

Re: [ANNOUNCE] New Committer: Ashutosh Mestry

2017-08-15 Thread Apoorv Naik 
Congratulations Ashutosh.





On 8/15/17, 10:21 AM, "Ayub Khan Pathan"  wrote:

>Congratulations, Ashutosh!
>
>On 8/13/17, 11:03 PM, "Madhan Neethiraj"  wrote:
>
>Atlas community,
>
> 
>
>Apache Atlas PMC has asked Ashutosh Mestry to become a committer and we 
> are pleased to announce that he has accepted.
>
> 
>
>Welcome Ashutosh, great to have you on board.
>
> 
>
>On behalf of Apache Atlas PMC,
>
> 
>
>Madhan
>
> 
>
> 
>
>
>

Re: [ANNOUNCE] New Committer: Mandy Chessell

2017-08-15 Thread Apoorv Naik 
Congratulations Mandy.




On 8/15/17, 10:21 AM, "Ayub Khan Pathan"  wrote:

>Congratulations, Mandy.
>
>On 8/15/17, 8:51 PM, "Madhan Neethiraj"  wrote:
>
>Atlas community,
>
> 
>
>Apache Atlas PMC has asked Mandy Chessell to become a committer and we are 
> pleased to announce that she has accepted.
>
> 
>
>Welcome Mandy, great to have you on board.
>
> 
>
>On behalf of Apache Atlas PMC,
>
> 
>
>Madhan
>
> 
>
> 
>
>
>

Review Request 61667: ATLAS-2044: In-memory filtering after index query

2017-08-15 Thread Apoorv Naik 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61667/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-2044
https://issues.apache.org/jira/browse/ATLAS-2044


Repository: atlas


Description
---

In-memory filtering is needed to weed out any false positive results from index 
query. False positive matches happen when the search string is present "as is" 
after a special token which is used by the indexer (solr, elasticsearch) to 
tokenize the string (during indexing as well as querying)

eg.

name = /a/b/c

search string "a", "b", "c" will match this name every time under equality, 
startsWith, endsWith check as it's present immediately after the special token.

The patch adds extra level of filtering on the the result set obtained from the 
index query.


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74197ca8 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
9cd83fb4 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 d556bf1a 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
b209ecb4 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/61667/diff/1/


Testing
---

REST and UI testing no longer show the false positive matches.

mvn clean package -Pdist (in progress)


Thanks,

Apoorv Naik

[jira] [Updated] (ATLAS-2044) InMemory filtering after index query

2017-08-15 Thread Apoorv Naik (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2044:
---
Attachment: 0001-ATLAS-2044-In-memory-filtering-for-correctness-after.patch

> InMemory filtering after index query
> 
>
> Key: ATLAS-2044
> URL: https://issues.apache.org/jira/browse/ATLAS-2044
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Attachments: 
> 0001-ATLAS-2044-In-memory-filtering-for-correctness-after.patch
>
>
> Currently the indexing and querying is done using a StandardTokenParser which 
> tokenizes the string on punctuations, delimiters as a result the following 
> test string will pass the equality match for the actual stored string
> test string: user
> stored val: /foo/bar/user or /foo/bar:user
> This patch adds another layer of filtering after the index query which 
> filters the vertices to ensure the correctness of results.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2044) InMemory filtering after index query

2017-08-15 Thread Apoorv Naik (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2044:
---
Attachment: (was: 
0001-ATLAS-2044-In-memory-filtering-for-correctness-after.patch)

> InMemory filtering after index query
> 
>
> Key: ATLAS-2044
> URL: https://issues.apache.org/jira/browse/ATLAS-2044
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>
> Currently the indexing and querying is done using a StandardTokenParser which 
> tokenizes the string on punctuations, delimiters as a result the following 
> test string will pass the equality match for the actual stored string
> test string: user
> stored val: /foo/bar/user or /foo/bar:user
> This patch adds another layer of filtering after the index query which 
> filters the vertices to ensure the correctness of results.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2044) InMemory filtering after index query

2017-08-15 Thread Apoorv Naik (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2044:
---
Attachment: 0001-ATLAS-2044-In-memory-filtering-for-correctness-after.patch

> InMemory filtering after index query
> 
>
> Key: ATLAS-2044
> URL: https://issues.apache.org/jira/browse/ATLAS-2044
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Attachments: 
> 0001-ATLAS-2044-In-memory-filtering-for-correctness-after.patch
>
>
> Currently the indexing and querying is done using a StandardTokenParser which 
> tokenizes the string on punctuations, delimiters as a result the following 
> test string will pass the equality match for the actual stored string
> test string: user
> stored val: /foo/bar/user or /foo/bar:user
> This patch adds another layer of filtering after the index query which 
> filters the vertices to ensure the correctness of results.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: apache-atlas-nightly #836

2017-08-15 Thread Apache Jenkins Server 
See 

--
[...truncated 683.60 KB...]
127.0.0.1 - - [15/Aug/2017:17:21:13 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:13 +] "DELETE 
/api/atlas/v2/entity/bulk/?guid=e5e91830-7330-461a-bc38-1551524ce933&guid=3a08211b-bf48-479e-82f5-ad2853c2133e
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:13 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:13 +] "DELETE 
/api/atlas/v2/entity/uniqueAttribute/type/hive_db_v2?attr:name=dbSapdDbyrjb%ED%97%A2%EF%A2%88%EC%AA%A9%EE%AE%98%E3%8A%9A%E1%96%AD%EA%AC%A9%EC%B8%B9%E6%9C%B1%E5%8E%8A
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:13 +] "DELETE 
/api/atlas/v2/entity/guid/random/classification/blah_trait HTTP/1.1" 400 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:13 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db_v2+where+name%3D'dbSapdDbyrjb'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:15 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db_v2+where+name%3D'dbSapdDbyrjb'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:15 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:15 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db_v2+where+name%3D'dbSapdDbyrjb'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:16 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:16 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:16 +] "PUT /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET 
/api/atlas/v2/entity/guid/326c6005-59b1-44a9-afaa-ade65f5d4856 HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET 
/api/atlas/v2/entity/uniqueAttribute/type/hive_db_v2?attr:name=dbSapdDbyrjb 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET /api/atlas/v2/entity/guid/blah 
HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET 
/api/atlas/entities/78eb841a-367c-4277-89f2-6421485deae4/audit?count=10 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET 
/api/atlas/v2/types/typedef/name/fm67KbBLvC HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "POST /api/atlas/v2/entity/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET 
/api/atlas/v2/entity/guid/e4f06c06-4d53-48de-9ca4-cf98361202f4 HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:17 +] "GET 
/api/atlas/discovery/search/dsl?query=from+hive_db_v1+name%3D%22dbOW2ZYWHb4x%22 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:18 +] "GET 
/api/atlas/discovery/search/dsl?query=from+blah HTTP/1.1" 400 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:18 +] "GET 
/api/atlas/discovery/search?query=* HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:18 +] "GET 
/api/atlas/discovery/search?query=from+hive_db_v1+name%3D%22dbOW2ZYWHb4x%22 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:19 +] "GET 
/api/atlas/discovery/search/gremlin?query=g.V.has('type',+'hive_table_v1').toList()
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:20 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:20 +] "POST /api/atlas/v2/types/typedefs/ 
HTTP/1.1" 409 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:20 +] "GET 
/api/atlas/v2/types/enumdef/name/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:20 +] "GET 
/api/atlas/v2/types/enumdef/guid/blah HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [15/Aug/2017:17:21:20 +] "GET 
/api/atlas/v2/types/struc

Re: [ANNOUNCE] New Committer: Mandy Chessell

2017-08-15 Thread Ayub Khan Pathan 
Congratulations, Mandy.

On 8/15/17, 8:51 PM, "Madhan Neethiraj"  wrote:

Atlas community,

 

Apache Atlas PMC has asked Mandy Chessell to become a committer and we are 
pleased to announce that she has accepted.

 

Welcome Mandy, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

Re: [ANNOUNCE] New Committer: Ashutosh Mestry

2017-08-15 Thread Ayub Khan Pathan 
Congratulations, Ashutosh!

On 8/13/17, 11:03 PM, "Madhan Neethiraj"  wrote:

Atlas community,

 

Apache Atlas PMC has asked Ashutosh Mestry to become a committer and we are 
pleased to announce that he has accepted.

 

Welcome Ashutosh, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

Re: [ANNOUNCE] New Committer: David Radley

2017-08-15 Thread Ayub Khan Pathan 
Congratulations, David.

On 8/13/17, 11:07 PM, "Madhan Neethiraj"  wrote:

Atlas community,

 

Apache Atlas PMC has asked David Radley to become a committer and we are 
pleased to announce that he has accepted.

 

Welcome David, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

Review Request 61665: ATLAS-2047: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

2017-08-15 Thread Ashutosh Mestry 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61665/
---

Review request for atlas, Madhan Neethiraj and Nixon Rodrigues.


Bugs: ATLAS-2047
https://issues.apache.org/jira/browse/ATLAS-2047


Repository: atlas


Description
---

Please refer to [ATLAS-2047](https://issues.apache.org/jira/browse/ATLAS-2047) 
for background and analysis.

**Implementation**

Please take a look at this scala code. This is _ShutdownableThread_. The thread 
does the job of handling all exceptions. Upon exception, it manages the 
_shutdownLatch_ (from yesterday’s bug fix) and gets out of the _isRunning_ loop.
```scala
  override def run(): Unit = {
info("Starting ")
try{
  while(isRunning.get()){
doWork()
  }
} catch{
  case e: Throwable =>
if(isRunning.get())
  error("Error due to ", e)
}
shutdownLatch.countDown()
info("Stopped ")
  }
```

Moved _try...catch_ block to leave exception handling to _ShutdownableThread_.


Diffs
-

  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 ef64c3b 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 a6f58e8 


Diff: https://reviews.apache.org/r/61665/diff/1/


Testing
---

**Unit tests**
Updated unit tests to reproduce the scenarios and verify the fix.

**Functional tests**
Verified regular notification scenarios.


Thanks,

Ashutosh Mestry

[jira] [Commented] (ATLAS-2047) NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

2017-08-15 Thread Ashutosh Mestry (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127514#comment-16127514
 ] 

Ashutosh Mestry commented on ATLAS-2047:


CC: [~nixonrodrigues]
CC: [~madhan.neethiraj]

> NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling 
> Logs Due to Incorrect Handling 
> 
>
> Key: ATLAS-2047
> URL: https://issues.apache.org/jira/browse/ATLAS-2047
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-2047-HookConsumer-ExceptionHandling.patch
>
>
> *Background*
> _KafkaConsumer_ is abstracted by _AtlasKafkaConsumer_. This is run using 
> _HookConsumer_ which is derived from _kafka.utils.ShutdownableThread_.
> The _ShutdownableThread_ manages the thread. It handles exceptions thrown in 
> the _doWork_ method and logs them.
> *Problem*
> Exception reported in the bug is thrown by _KafkaConsumer_ is handed by 
> _HookConsumer_. Exception is logged but the thread keeps running. In cases 
> where Kafa is in irrecoverable state, this behavior ends up filling up logs 
> and providing no value in return.
> *Solution*
> Let _ShutdownableThread_ hand exceptions thrown by _KafkaConsumer_. Stop the 
> thread if _KafkaConsumer_ suffers from irrecoverable error. This will avoid 
> the situation described in the problem section.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2047) NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

2017-08-15 Thread Ashutosh Mestry (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-2047:
---
Attachment: ATLAS-2047-HookConsumer-ExceptionHandling.patch

> NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling 
> Logs Due to Incorrect Handling 
> 
>
> Key: ATLAS-2047
> URL: https://issues.apache.org/jira/browse/ATLAS-2047
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk, 0.8-incubating
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
> Fix For: trunk
>
> Attachments: ATLAS-2047-HookConsumer-ExceptionHandling.patch
>
>
> *Background*
> _KafkaConsumer_ is abstracted by _AtlasKafkaConsumer_. This is run using 
> _HookConsumer_ which is derived from _kafka.utils.ShutdownableThread_.
> The _ShutdownableThread_ manages the thread. It handles exceptions thrown in 
> the _doWork_ method and logs them.
> *Problem*
> Exception reported in the bug is thrown by _KafkaConsumer_ is handed by 
> _HookConsumer_. Exception is logged but the thread keeps running. In cases 
> where Kafa is in irrecoverable state, this behavior ends up filling up logs 
> and providing no value in return.
> *Solution*
> Let _ShutdownableThread_ hand exceptions thrown by _KafkaConsumer_. Stop the 
> thread if _KafkaConsumer_ suffers from irrecoverable error. This will avoid 
> the situation described in the problem section.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2047) NotificationHookConsumer: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

2017-08-15 Thread Ashutosh Mestry (JIRA) 
Ashutosh Mestry created ATLAS-2047:
--

 Summary: NotificationHookConsumer: Exception Thrown by Kafka 
Consumer Ends up Filling Logs Due to Incorrect Handling 
 Key: ATLAS-2047
 URL: https://issues.apache.org/jira/browse/ATLAS-2047
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.8-incubating, trunk
Reporter: Ashutosh Mestry
Assignee: Ashutosh Mestry
 Fix For: trunk


*Background*

_KafkaConsumer_ is abstracted by _AtlasKafkaConsumer_. This is run using 
_HookConsumer_ which is derived from _kafka.utils.ShutdownableThread_.

The _ShutdownableThread_ manages the thread. It handles exceptions thrown in 
the _doWork_ method and logs them.

*Problem*
Exception reported in the bug is thrown by _KafkaConsumer_ is handed by 
_HookConsumer_. Exception is logged but the thread keeps running. In cases 
where Kafa is in irrecoverable state, this behavior ends up filling up logs and 
providing no value in return.

*Solution*
Let _ShutdownableThread_ hand exceptions thrown by _KafkaConsumer_. Stop the 
thread if _KafkaConsumer_ suffers from irrecoverable error. This will avoid the 
situation described in the problem section.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] New Committer: Ashutosh Mestry

2017-08-15 Thread David Radley 
Congratulations Ashutosh,



From:   Madhan Neethiraj 
To: 
Cc: "ames...@apache.org" 
Date:   13/08/2017 18:33
Subject:[ANNOUNCE] New Committer: Ashutosh Mestry



Atlas community,

 

Apache Atlas PMC has asked Ashutosh Mestry to become a committer and we 
are pleased to announce that he has accepted.

 

Welcome Ashutosh, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

 

 




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [ANNOUNCE] New Committer: Mandy Chessell

2017-08-15 Thread Ernie Ostic 
Congrats Mandy!Thanks for all you contributions to the models, the Virtual 
Connector and beyond.

Ernie



Sent from IBM Verse


   Madhan Neethiraj --- [ANNOUNCE] New Committer: Mandy Chessell --- 
From:"Madhan Neethiraj" 
To:dev@atlas.apache.orgCc:mandy@apache.orgDate:Tue, Aug 15, 
2017 11:21 AMSubject:[ANNOUNCE] New Committer: Mandy Chessell
  
Atlas community, Apache Atlas PMC has asked Mandy Chessell to become a 
committer and we are pleased to announce that she has accepted. Welcome Mandy, 
great to have you on board. On behalf of Apache Atlas PMC, Madhan

Re: [ANNOUNCE] New Committer: Mandy Chessell

2017-08-15 Thread Mandy Chessell 
Thanks Madhan - I am delighted this has happened and love working on the Apache 
Atlas project.  I believe we are building an important capability for the 
industry.

> On 15 Aug 2017, at 16:21, Madhan Neethiraj  wrote:
> 
> Atlas community,
>  
> Apache Atlas PMC has asked Mandy Chessell to become a committer and we are 
> pleased to announce that she has accepted.
>  
> Welcome Mandy, great to have you on board.
>  
> On behalf of Apache Atlas PMC,
>  
> Madhan
>  
>

[jira] [Created] (ATLAS-2046) UI Regression : Columns Filter not listing all attributes of a datatype

2017-08-15 Thread Sharmadha Sainath (JIRA) 
Sharmadha Sainath created ATLAS-2046:


 Summary: UI Regression : Columns Filter not listing all attributes 
of a datatype
 Key: ATLAS-2046
 URL: https://issues.apache.org/jira/browse/ATLAS-2046
 Project: Atlas
  Issue Type: Bug
  Components: atlas-webui
Affects Versions: 0.9-incubating
Reporter: Sharmadha Sainath
Priority: Critical
 Attachments: Screen Shot 2017-08-15 at 8.32.18 PM.png

"Columns" filter button doesn't list down all the attributes based on the 
datatype . It shows the default set of attributes only. 

Attached the screenshot for search typename =  hive_table



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[ANNOUNCE] New Committer: Mandy Chessell

2017-08-15 Thread Madhan Neethiraj 
Atlas community,

 

Apache Atlas PMC has asked Mandy Chessell to become a committer and we are 
pleased to announce that she has accepted.

 

Welcome Mandy, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

[jira] [Commented] (ATLAS-1698) Create Glossary OMAS API

2017-08-15 Thread Nigel Jones (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127325#comment-16127325
 ] 

Nigel Jones commented on ATLAS-1698:


I thought I'd try and summarise some of my thoughts around OMAS design in 
general (not specific to glossary or indeed GAF) through a series of statements 
at 
https://cwiki.apache.org/confluence/display/ATLAS/Standards+for+OMAS+interfaces

Agree or not, hopefully this could be the start of documenting our "standards" 
of how to define an OMAS interface?

> Create Glossary OMAS API
> 
>
> Key: ATLAS-1698
> URL: https://issues.apache.org/jira/browse/ATLAS-1698
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: David Radley
>Assignee: David Radley
>  Labels: VirtualDataConnector
> Attachments: apidocs-5.zip, Atlas Glossary OMAS API proposal v1.0 
> .pdf, Atlas Glossary OMAS API proposal v1.1.pdf, Atlas Glossary OMAS API 
> proposal v1.2.pdf
>
>
> The Glossary OMAS provides a specialized API for glossary applications to 
> retrieve and store their glossary metadata and link assets of different types 
> to these glossary entries.
> The Glossary OMAS makes heavy use of the Area 2 open metadata model.  See 
> https://cwiki.apache.org/confluence/display/ATLAS/Area+2+-+Glossary



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1691) OMAS Interfaces for Atlas

2017-08-15 Thread Nigel Jones (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127323#comment-16127323
 ] 

Nigel Jones commented on ATLAS-1691:


I thought I'd start putting together a set of statements that relate to what 
OMAS is and how we build it.
See 
https://cwiki.apache.org/confluence/display/ATLAS/Standards+for+OMAS+interfaces
Comments welcome ... it's just a start..

> OMAS Interfaces for Atlas
> -
>
> Key: ATLAS-1691
> URL: https://issues.apache.org/jira/browse/ATLAS-1691
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nigel Jones
>Assignee: Mandy Chessell
>  Labels: VirtualDataConnector
>
> The Open Metadata Access Services (OMAS) provide consumer specific interfaces 
> to the metadata repository.   The hope is to have a community of vendor tools 
> developing around each OMAS interface.   The OMAS interfaces are independent 
> from one another to allow each community to evolve their own API at their 
> pace.
> Related Jira will cover the list of OMAS interfaces that are oriented to 
> particular tools & usage.
> Each OMAS JIRA needs to cover:
> * The Java and REST API for the OMAS call-response API
> * The appropriate OMAS Event Publisher and Listener



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-2045) rename the shipped Atlas model files

2017-08-15 Thread David Radley (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-2045:

Attachment: ATLAS-2045.patch

> rename the shipped Atlas model files
> 
>
> Key: ATLAS-2045
> URL: https://issues.apache.org/jira/browse/ATLAS-2045
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: ATLAS-2045.patch
>
>
> This change is to rename the shipped Atlas model files. The currently start 
> 0010,0020,0030.
> We will not rename the file 0010 base model file. 
> We will rename 0020... -> 1020... , 0030 -> 1030 etc. 
> This renumbering will create space for new models - for example those in 
> ATLAS_1836 to be added. 
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 61657: ATLAS-2045 Renumber shipped Atlas models

2017-08-15 Thread David Radley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61657/
---

Review request for atlas, Madhan Neethiraj and Mandy Chessell.


Repository: atlas


Description
---

ATLAS-2045 Renumber shipped Atlas models


Diffs
-

  addons/models/0020-fs_model.json  
  addons/models/0030-hive_model.json  
  addons/models/0040-sqoop_model.json  
  addons/models/0050-falcon_model.json  
  addons/models/0060-hbase_model.json  
  addons/models/0070-kafka_model.json  
  addons/models/0080-storm_model.json  
  
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
 42f70111d513217b8fbe73e1416c6c94e91a1b7d 


Diff: https://reviews.apache.org/r/61657/diff/1/


Testing
---

Ran the junits


Thanks,

David Radley

[jira] [Assigned] (ATLAS-2045) rename the shipped Atlas model files

2017-08-15 Thread David Radley (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley reassigned ATLAS-2045:
---

Assignee: David Radley

> rename the shipped Atlas model files
> 
>
> Key: ATLAS-2045
> URL: https://issues.apache.org/jira/browse/ATLAS-2045
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> This change is to rename the shipped Atlas model files. The currently start 
> 0010,0020,0030.
> We will not rename the file 0010 base model file. 
> We will rename 0020... -> 1020... , 0030 -> 1030 etc. 
> This renumbering will create space for new models - for example those in 
> ATLAS_1836 to be added. 
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-2045) rename the shipped Atlas model files

2017-08-15 Thread David Radley (JIRA) 
David Radley created ATLAS-2045:
---

 Summary: rename the shipped Atlas model files
 Key: ATLAS-2045
 URL: https://issues.apache.org/jira/browse/ATLAS-2045
 Project: Atlas
  Issue Type: Improvement
Reporter: David Radley


This change is to rename the shipped Atlas model files. The currently start 
0010,0020,0030.
We will not rename the file 0010 base model file. 
We will rename 0020... -> 1020... , 0030 -> 1030 etc. 

This renumbering will create space for new models - for example those in 
ATLAS_1836 to be added. 
 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: apache-atlas-nightly #835

2017-08-15 Thread Apache Jenkins Server 
See 


Changes:

[david_radley] Add Javadoc format to class summaries

--
[...truncated 607.13 KB...]
Caused by: java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.integration.TypedefsJerseyResourceIT.testUpdate(TypedefsJerseyResourceIT.java:133)

testDuplicateSubmit(org.apache.atlas.web.integration.TypesJerseyResourceIT)  
Time elapsed: 0.003 sec  <<< FAILURE!
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: 
Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testDuplicateSubmit(TypesJerseyResourceIT.java:102)
Caused by: java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testDuplicateSubmit(TypesJerseyResourceIT.java:102)

testGetDefinitionForNonexistentType(org.apache.atlas.web.integration.TypesJerseyResourceIT)
  Time elapsed: 0.001 sec  <<< FAILURE!
org.testng.TestException: 

Expected exception org.apache.atlas.AtlasServiceException but got 
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: 
Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testGetDefinitionForNonexistentType(TypesJerseyResourceIT.java:167)
Caused by: java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testGetDefinitionForNonexistentType(TypesJerseyResourceIT.java:167)

testGetTraitNames(org.apache.atlas.web.integration.TypesJerseyResourceIT)  Time 
elapsed: 0.003 sec  <<< FAILURE!
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: 
Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.addTraits(TypesJerseyResourceIT.java:226)
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testGetTraitNames(TypesJerseyResourceIT.java:188)
Caused by: java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.addTraits(TypesJerseyResourceIT.java:226)
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testGetTraitNames(TypesJerseyResourceIT.java:188)

testListTypesByFilter(org.apache.atlas.web.integration.TypesJerseyResourceIT)  
Time elapsed: 0.005 sec  <<< FAILURE!
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: 
Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testListTypesByFilter(TypesJerseyResourceIT.java:206)
Caused by: java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testListTypesByFilter(TypesJerseyResourceIT.java:206)

testUpdate(org.apache.atlas.web.integration.TypesJerseyResourceIT)  Time 
elapsed: 0.002 sec  <<< FAILURE!
com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: 
Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testUpdate(TypesJerseyResourceIT.java:117)
Caused by: java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.integration.TypesJerseyResourceIT.testUpdate(TypesJerseyResourceIT.java:117)

testSimpleLoginForValidUser(org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT)
  Time elapsed: 0.001 sec  <<< FAILURE!
java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT.testSimpleLoginForValidUser(AtlasAuthenticationSimpleFilterIT.java:45)

testSimpleLoginAndAuthorizationWithValidCrendentialsAndInvalidAccessToResource(org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT)
  Time elapsed: 0.001 sec  <<< FAILURE!
java.lang.AssertionError: Failed with exception Connection refused
at 
org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT.testSimpleLoginAndAuthorizationWithValidCrendentialsAndInvalidAccessToResource(AtlasAuthenticationSimpleFilterIT.java:78)

testAccessforUnauthenticatedResource(org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT)
  Time elapsed: 0 sec  <<< FAILURE!
java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT.testAccessforUnauthenticatedResource(AtlasAuthenticationSimpleFilterIT.java:57)

testSimpleLoginWithInvalidCrendentials(org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT)
  Time elapsed: 0 sec  <<< FAILURE!
java.net.ConnectException: Connection refused
at 
org.apache.atlas.web.filters.AtlasAuthenticationSimpleFilterIT.testSimpleLoginWithInvalidCrendentials(AtlasAuthenticationSimpleFilterIT.java:93)


Results :

Failed tests: 
  QuickStartV2IT.runQuickStart:49 » AtlasService Metadata service API 
org.apache...
  
NotificationHookConsumerIT.testDeleteByQualifiedName:184->BaseResourceIT.waitFor:626
 » 
  
NotificationHookConsumerIT.testUpdateEntityPartial:127

[jira] [Commented] (ATLAS-2003) Add Javadoc format to class summaries

2017-08-15 Thread David Radley (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-2003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127181#comment-16127181
 ] 

David Radley commented on ATLAS-2003:
-

Thanks [~rding] committed fix 
http://git-wip-us.apache.org/repos/asf/atlas/commit/5e638c24

> Add Javadoc format to class summaries
> -
>
> Key: ATLAS-2003
> URL: https://issues.apache.org/jira/browse/ATLAS-2003
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8-incubating
>Reporter: Richard Ding
>Assignee: Richard Ding
>Priority: Minor
> Fix For: 0.9-incubating
>
> Attachments: after javadoc formating.png, ATLAS-2003.patch, before 
> javadoc formating.png
>
>
> Java class summary needs to be formatted with HTML tags so that the generated 
> Javadoc can be properly displayed in browsers.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 61568: ATLAS-2003: Add Javadoc format to class summaries

2017-08-15 Thread David Radley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61568/#review182940
---


Ship it!




Ship It!

- David Radley


On Aug. 10, 2017, 6:15 p.m., Richard Ding wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61568/
> ---
> 
> (Updated Aug. 10, 2017, 6:15 p.m.)
> 
> 
> Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2003
> https://issues.apache.org/jira/browse/ATLAS-2003
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch added Javadoc format to the following class summaries: 
> ```
> InMemoryJAASConfiguration.java 
> TitanGraphQuery.java   
> AtlasRelationshipDef.java  
> NotificationInterface.java 
> ExpandAndsOptimization.java 
> HBaseBasedAuditRepository.java   
> ```
> 
> 
> Diffs
> -
> 
>   
> common/src/main/java/org/apache/atlas/security/InMemoryJAASConfiguration.java 
> c65048fa 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/TitanGraphQuery.java
>  c38f6cc8 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
> c17e875d 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
>  ef8ee275 
>   
> repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpandAndsOptimization.java
>  7cf97110 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  06e518ee 
> 
> 
> Diff: https://reviews.apache.org/r/61568/diff/1/
> 
> 
> Testing
> ---
> 
> Verified with Javadoc build.
> 
> 
> Thanks,
> 
> Richard Ding
> 
>

Re: [ANNOUNCE] New Committer: David Radley

2017-08-15 Thread Hemanth Yamijala 
Congratulations, David.

From: Madhan Neethiraj 
Sent: Sunday, August 13, 2017 11:07 PM
To: dev@atlas.apache.org
Cc: David Radley
Subject: [ANNOUNCE] New Committer: David Radley

Atlas community,



Apache Atlas PMC has asked David Radley to become a committer and we are 
pleased to announce that he has accepted.



Welcome David, great to have you on board.



On behalf of Apache Atlas PMC,



Madhan

Re: [ANNOUNCE] New Committer: Ashutosh Mestry

2017-08-15 Thread Hemanth Yamijala 
Congratulations, Ashutosh!

From: Madhan Neethiraj 
Sent: Sunday, August 13, 2017 11:03 PM
To: dev@atlas.apache.org
Cc: ames...@apache.org
Subject: [ANNOUNCE] New Committer: Ashutosh Mestry

Atlas community,



Apache Atlas PMC has asked Ashutosh Mestry to become a committer and we are 
pleased to announce that he has accepted.



Welcome Ashutosh, great to have you on board.



On behalf of Apache Atlas PMC,



Madhan

[jira] [Updated] (ATLAS-1839) Area 2 of the open metadata model

2017-08-15 Thread David Radley (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1839:

Attachment: 0230Terms.json

> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: David Radley
>  Labels: OpenMetadata, VirtualDataConnector
> Attachments: 0210Glossary.json, 0220CategoryHierarchy.json, 
> 0230Terms.json, 0240Dictionary.json, 0250RelatedTerms.json, 
> 0260Contexts.json, 0270SemanticAssignment.json, 0280SpineObjects.json
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1839) Area 2 of the open metadata model

2017-08-15 Thread David Radley (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1839:

Attachment: (was: 0230Terms.json)

> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: David Radley
>  Labels: OpenMetadata, VirtualDataConnector
> Attachments: 0210Glossary.json, 0220CategoryHierarchy.json, 
> 0230Terms.json, 0240Dictionary.json, 0250RelatedTerms.json, 
> 0260Contexts.json, 0270SemanticAssignment.json, 0280SpineObjects.json
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1839) Area 2 of the open metadata model

2017-08-15 Thread David Radley (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1839:

Attachment: 0230Terms.json

> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: David Radley
>  Labels: OpenMetadata, VirtualDataConnector
> Attachments: 0210Glossary.json, 0220CategoryHierarchy.json, 
> 0230Terms.json, 0240Dictionary.json, 0250RelatedTerms.json, 
> 0260Contexts.json, 0270SemanticAssignment.json, 0280SpineObjects.json
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1839) Area 2 of the open metadata model

2017-08-15 Thread David Radley (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1839:

Attachment: (was: 0230Terms.json)

> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: David Radley
>  Labels: OpenMetadata, VirtualDataConnector
> Attachments: 0210Glossary.json, 0220CategoryHierarchy.json, 
> 0230Terms.json, 0240Dictionary.json, 0250RelatedTerms.json, 
> 0260Contexts.json, 0270SemanticAssignment.json, 0280SpineObjects.json
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)