[jira] [Updated] (ATLAS-1946) LDAPS(LDAP over SSL) support in Atlas

2017-07-11 Thread Ayub Pathan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ayub Pathan updated ATLAS-1946:
---
Issue Type: Improvement  (was: Bug)

> LDAPS(LDAP over SSL) support in Atlas
> -
>
> Key: ATLAS-1946
> URL: https://issues.apache.org/jira/browse/ATLAS-1946
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-intg
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>
> Add support for LDAPS in Atlas. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1946) LDAPS(LDAP over SSL) support in Atlas

2017-07-11 Thread Ayub Pathan (JIRA) 
Ayub Pathan created ATLAS-1946:
--

 Summary: LDAPS(LDAP over SSL) support in Atlas
 Key: ATLAS-1946
 URL: https://issues.apache.org/jira/browse/ATLAS-1946
 Project: Atlas
  Issue Type: Bug
  Components: atlas-intg
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan


Add support for LDAPS in Atlas. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1945) UI : Moving all plugins to fix release version

2017-07-11 Thread Keval Bhatt (JIRA) 
Keval Bhatt created ATLAS-1945:
--

 Summary: UI : Moving all plugins to fix release version
 Key: ATLAS-1945
 URL: https://issues.apache.org/jira/browse/ATLAS-1945
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.9-incubating
Reporter: Keval Bhatt
Assignee: Keval Bhatt
Priority: Blocker






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1944) NotificationHookConsumer throws exception while shutting down the consumer thread

2017-07-11 Thread Ayub Pathan (JIRA) 
Ayub Pathan created ATLAS-1944:
--

 Summary: NotificationHookConsumer throws exception while shutting 
down the consumer thread
 Key: ATLAS-1944
 URL: https://issues.apache.org/jira/browse/ATLAS-1944
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.9-incubating
Reporter: Ayub Pathan
Priority: Critical
 Fix For: 0.9-incubating


NotificationHookConsumer throws below exception while shutting down the 
consumer thread, this issue is possibly after this commit.
https://github.com/apache/incubator-atlas/commit/0e7f8ea4603c858cc295259bbd1a22314b732f62

CC [~nrodrigues]

{noformat}
2017-07-12 01:26:09,743 WARN  - [pool-1-thread-1:] ~ Error stopping service 
org.apache.atlas.notification.NotificationHookConsumer (Services:69)
java.util.ConcurrentModificationException: KafkaConsumer is not safe for 
multi-threaded access
at 
org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1557)
at 
org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1467)
at 
org.apache.atlas.kafka.AtlasKafkaConsumer.close(AtlasKafkaConsumer.java:88)
at 
org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.stop(NotificationHookConsumer.java:384)
at 
org.apache.atlas.notification.NotificationHookConsumer.stopConsumerThreads(NotificationHookConsumer.java:172)
at 
org.apache.atlas.notification.NotificationHookConsumer.stop(NotificationHookConsumer.java:155)
at org.apache.atlas.service.Services.stop(Services.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeDestroyMethods(InitDestroyAnnotationBeanPostProcessor.java:325)
at 
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeDestruction(InitDestroyAnnotationBeanPostProcessor.java:154)
at 
org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253)
at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578)
at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554)
at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961)
at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523)
at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968)
at 
org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1033)
at 
org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1009)
at 
org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:961)
at 
org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:583)
at 
org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:116)
at 
org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:808)
at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:457)
at 
org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:842)
at 
org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:215)
at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:529)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:143)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:162)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
at org.eclipse.jetty.server.Server.doStop(Server.java:456)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
{noformat}



--
This message was

[jira] [Commented] (ATLAS-1874) V2 DSL search query does not support for count() but v1 do.

2017-07-11 Thread Ayub Pathan (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083284#comment-16083284
 ] 

Ayub Pathan commented on ATLAS-1874:


[~grahamwallis] DSL query works fine for the above query you mentioned, it's 
only that UI does not display anything, because query is requesting for 
specific attribute (in this case, owner) of that entity. Please check the below 
response for same query.

Response for query: "`hive_table` where db.name=\"default\" select owner"
{noformat}
{  
   "queryType":"DSL",
   "queryText":"`hive_table` where db.name=\"default\" select owner",
   "attributes":{  
  "name":[  
 "owner"
  ],
  "values":[  
 [  
"hive"
 ]
  ]
   }
}
{noformat}

> V2 DSL search query does not support for count() but v1 do.
> ---
>
> Key: ATLAS-1874
> URL: https://issues.apache.org/jira/browse/ATLAS-1874
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8.1-incubating
>Reporter: Ayub Pathan
> Fix For: 0.9-incubating, 0.8.1-incubating
>
>
> v1 DSL query supports count()
> {noformat}
> curl -u admin:admin 
> 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?query=hive_table%20where%20db.name%3D%22default%22%20select%20count()%20as%20%27count%27'
>  | python -m json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100   5370   5370 0339  0 --:--:--  0:00:01 --:--:--   339
> {
> "count": 1,
> "dataType": {
> "attributeDefinitions": [
> {
> "dataTypeName": "long",
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "count",
> "reverseAttributeName": null
> }
> ],
> "typeDescription": null,
> "typeName": "__tempQueryResultStruct286",
> "typeVersion": "1.0"
> },
> "query": "hive_table where db.name=\"default\" select count() as 'count'",
> "queryType": "dsl",
> "requestId": "pool-2-thread-9 - 47389e3f-bbf1-4209-8e50-8a3235a7e5a9",
> "results": [
> {
> "$typeName$": "__tempQueryResultStruct286",
> "count": 68
> }
> ]
> }
> {noformat}
> v2 DSL search query does not
> {noformat}
> curl -u admin:admin 
> 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25=true=where+db.name%3D%22default%22+select+count()+as+%27count%27=hive_table&_=1497434602692'
>  | python -m json.tool
> {
> "queryText": "`hive_table` where db.name=\"default\" select count() as 
> 'count'",
> "queryType": "DSL"
> }
> {noformat}
> *From the initial analysis, it seems like, V2 API response does not have the 
> count attribute which is cause for this failure.* might want to consider 
> adding the count in V2?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1943) Fix IT failure, lineage due to incorrect inverse reference check using relationship

2017-07-11 Thread Sarath Subramanian (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1943:
--
Summary: Fix IT failure, lineage due to incorrect inverse reference check 
using relationship  (was: Fix IT failure due to incorrect inverse reference 
check using relationship)

> Fix IT failure, lineage due to incorrect inverse reference check using 
> relationship
> ---
>
> Key: ATLAS-1943
> URL: https://issues.apache.org/jira/browse/ATLAS-1943
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Critical
> Attachments: ATLAS-1943.1.patch
>
>
> Fix the following IT failures due to
> Failed tests: 
>   EntityJerseyResourceIT.testCompleteUpdate:952 » AtlasService Metadata 
> service ...
>   EntityJerseyResourceIT.testPartialUpdate:877 » AtlasService Metadata 
> service A...
> After creating a relationship for inverse reference attribute, call is made 
> to create relationship for ownedRef attribute. But relationship for ownedRef 
> has already been created and we see exception in log - “relationship already 
> exists”. 
> For e.g.: after creating a relationship for hive_storagedesc.table, a new 
> relationship to create hive_table.sd is attempted – but fails since 
> relationship already exists (already created).  Added a new method 
> getOrCreate(relationship) - If a relationship already exists between two 
> entities on an attribute retrieve it, if not create one. Validated the patch 
> by creating hive_table, views – lineage are rendered fine.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 60783: [ATLAS-1943] : Fix IT failure due to incorrect inverse reference check using relationship

2017-07-11 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60783/
---

Review request for atlas and Madhan Neethiraj.


Bugs: ATLAS-1943
https://issues.apache.org/jira/browse/ATLAS-1943


Repository: atlas


Description
---

Fix the following IT failures due to
Failed tests: 
EntityJerseyResourceIT.testCompleteUpdate:952 » AtlasService Metadata service 
...
EntityJerseyResourceIT.testPartialUpdate:877 » AtlasService Metadata service 
A...
After creating inverse reference, call is made to create relationship for 
ownedRef attribute. This call fails when an edge already exists. The edge 
exists as it was already created. Added getOrCreate relationship method to get 
a relationship if it exists, if not create a new relationship.


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasRelationshipStore.java
 341711ac 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 8d9e4be8 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 d5c1e862 


Diff: https://reviews.apache.org/r/60783/diff/1/


Testing
---

mvn clean package suceeded.


Thanks,

Sarath Subramanian

[jira] [Updated] (ATLAS-1943) Fix IT failure due to incorrect inverse reference check using relationship

2017-07-11 Thread Sarath Subramanian (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1943:
--
Attachment: ATLAS-1943.1.patch

> Fix IT failure due to incorrect inverse reference check using relationship
> --
>
> Key: ATLAS-1943
> URL: https://issues.apache.org/jira/browse/ATLAS-1943
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Critical
> Attachments: ATLAS-1943.1.patch
>
>
> Fix the following IT failures due to
> Failed tests: 
>   EntityJerseyResourceIT.testCompleteUpdate:952 » AtlasService Metadata 
> service ...
>   EntityJerseyResourceIT.testPartialUpdate:877 » AtlasService Metadata 
> service A...
> After creating inverse reference, call is made to create relationship for 
> ownedRef attribute. This call fails when an edge already exists. The edge 
> exists as it was already created. Added getOrCreate relationship method to 
> get a relationship if it exists, if not create a new relationship.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Build failed in Jenkins: apache-atlas-nightly #799

2017-07-11 Thread Apache Jenkins Server 
See 


Changes:

[madhan] ATLAS-1733: updated atlas_stop.py script to work in Windows environment

--
[...truncated 543.40 KB...]
127.0.0.1 - - [11/Jul/2017:17:16:05 +] "GET 
/api/atlas/discovery/search?query=PII_v1=10=0 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:06 +] "GET 
/api/atlas/discovery/search?query=%60Log+Data_v1%60=10=0 HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:07 +] "GET 
/api/atlas/discovery/search?query=Table_v1+where+name%3D%22sales_fact%22,+columns=10=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:08 +] "GET 
/api/atlas/discovery/search?query=Table_v1+where+name%3D%22sales_fact%22,+columns+as+column+select+column.name,+column.dataType,+column.comment=10=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:09 +] "GET 
/api/atlas/discovery/search?query=from+DataSet=10=0 HTTP/1.1" 200 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:10 +] "GET 
/api/atlas/discovery/search?query=from+Process=10=0 HTTP/1.1" 200 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=View_v1=qualifiedName=product_dim_view 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=product_dim 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=LoadProcess_v1=qualifiedName=loadSalesDaily
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=sales_fact 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=time_dim 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=sales_fact_daily_mv
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=sales_fact 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=time_dim 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=sales_fact_daily_mv
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:11 +] "GET 
/api/atlas/lineage/hive/table/sales_fact_daily_mv/inputs/graph HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:13 +] "GET 
/api/atlas/entities?type=DB_v1=name=Sales HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:13 +] "GET 
/api/atlas/entities?type=Table_v1=qualifiedName=sales_fact 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:13 +] "GET 
/api/atlas/entities?type=DB_v1=name=Sales HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/tableType HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/serdeType HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/classification HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/pii HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/phi HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/pci HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/sox HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/sec HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/finance HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/Fact HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/ETL HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/Dimension HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/Metric HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/hive_db_v1 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [11/Jul/2017:17:16:14 +] "GET 
/api/atlas/v2/types/typedef/name/hive_column_v1 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 -

[jira] [Updated] (ATLAS-1902) UI: Allow user to add more columns in search table

2017-07-11 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1902:
---
Summary: UI: Allow user to add more columns in search table  (was: UI: 
Allow user to add more column in search table)

> UI: Allow user to add more columns in search table
> --
>
> Key: ATLAS-1902
> URL: https://issues.apache.org/jira/browse/ATLAS-1902
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>
> In basic search after searching for type user can add more column based on 
> type selection. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (ATLAS-1733) No SIGKILL in python signal library on Windows

2017-07-11 Thread Madhan Neethiraj (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-1733.
-
   Resolution: Fixed
Fix Version/s: 0.8.1-incubating
   0.9-incubating

Thanks [~grahamwallis] for the fix; [~davidrad] for reviewing the patch.

Committed to the following branches:
 - master: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/ad398393
 - 0.8-incubating: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/13d5bfbf

> No SIGKILL in python signal library on Windows
> --
>
> Key: ATLAS-1733
> URL: https://issues.apache.org/jira/browse/ATLAS-1733
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
> Environment: Windows 7
>Reporter: Graham Wallis
>Assignee: Graham Wallis
>Priority: Minor
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1733.patch, ATLAS-1733.patch
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> [I am new to Atlas and this is my first JIRA issue so apologies if I have 
> omitted any necessary information or steps.]
> There is no SIGKILL signal in the python signal library on Windows, so the 
> import in atlas_stop.py fails.
> There are two ways I can suggest that we fix it:
> 1) introduce platform-specific code into atlas_stop.py
> 2) wrap the python os.kill function
> My preference is for the former as it is a small and localized change. 
> In either case the use of SIGTERM should be sufficient on Windows, as the 
> existing python os.kill() function will open the process and terminate it, 
> which is supposed to be equivalent to a SIGKILL (-9) on Linux.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1902) UI: Allow user to add more columns in search table

2017-07-11 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1902:
---
Description: In basic search after searching for a type, user can add more 
columns based on type selected  (was: In basic search after searching for type 
user can add more column based on type selection. )

> UI: Allow user to add more columns in search table
> --
>
> Key: ATLAS-1902
> URL: https://issues.apache.org/jira/browse/ATLAS-1902
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>
> In basic search after searching for a type, user can add more columns based 
> on type selected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 60773: ATLAS-1902 : UI: Allow user to add more columns in search table

2017-07-11 Thread keval bhatt 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60773/
---

Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-1902
https://issues.apache.org/jira/browse/ATLAS-1902


Repository: atlas


Description
---

In basic search after searching for a type, user can add more columns based on 
type selected


Diffs
-

  dashboardv2/gruntfile.js c0e315f 
  dashboardv2/package.json 59889d3 
  dashboardv2/public/css/scss/main.scss a608c95 
  dashboardv2/public/css/scss/override.scss 6821d77 
  dashboardv2/public/index.html b460686 
  dashboardv2/public/js/main.js 9235d28 
  dashboardv2/public/js/utils/CommonViewFunction.js de46667 
  dashboardv2/public/js/utils/TableLayout.js f6183a9 
  dashboardv2/public/js/views/audit/AuditTableLayoutView.js f650b1c 
  dashboardv2/public/js/views/audit/CreateAuditTableLayoutView.js 34c56cb 
  dashboardv2/public/js/views/detail_page/DetailPageLayoutView.js d44962f 
  dashboardv2/public/js/views/entity/EntityDetailTableLayoutView.js 79f7807 
  dashboardv2/public/js/views/search/SearchLayoutView.js c77d8a7 
  dashboardv2/public/js/views/search/SearchResultLayoutView.js 5236751 


Diff: https://reviews.apache.org/r/60773/diff/1/


Testing
---

Did on round of sanity test for atlas UI.


Thanks,

keval bhatt

[jira] [Updated] (ATLAS-1941) HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect one

2017-07-11 Thread Ayub Pathan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ayub Pathan updated ATLAS-1941:
---
Priority: Minor  (was: Major)

> HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to 
> incorrect one
> --
>
> Key: ATLAS-1941
> URL: https://issues.apache.org/jira/browse/ATLAS-1941
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Priority: Minor
> Fix For: 0.9-incubating
>
>
> While Atlas startup, HBaseStoreManager is copying host list from 
> root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect 
> zookeeper host as well.
> {noformat}
> 2017-07-11 12:13:51,341 INFO  - [main:] ~ Not running setup per configuration 
> atlas.server.run.setup.on.start. (SetupSteps:189)
> 2017-07-11 12:13:52,332 INFO  - [main:] ~ JSR-330 'javax.inject.Inject' 
> annotation found and supported for autowiring 
> (AutowiredAnnotationBeanPostProcessor:155)
> 2017-07-11 12:13:53,038 INFO  - [main:] ~ Instantiated HBase compatibility 
> layer supporting runtime HBase version 1.1.2.2.6.2.0-71: 
> com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 
> (HBaseCompatLoader:69)
> 2017-07-11 12:13:53,108 INFO  - [main:] ~ Copied host list from 
> root.storage.hostname to hbase.zookeeper.quorum: 
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site
>  (HBaseStoreManager:320)
> {noformat}
> In the above log, the correct zookeeper hosts are 
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site.
>  But, as we can see HBaseStoreManager is adding 
> ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect.
> As an outcome of this, zookeeper connection to this host fails.
> {noformat}
> 2017-07-11 12:13:53,399 INFO  - 
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ 
> Opening socket connection to server 
> ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will 
> attempt to SASL-authenticate using Login Context section 'Client' 
> (ClientCnxn:1019)
> 2017-07-11 12:13:53,412 WARN  - 
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ 
> Session 0x0 for server null, unexpected error, closing socket connection and 
> attempting reconnect (ClientCnxn:1146)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at 
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at 
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at 
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1874) V2 DSL search query does not support for count() but v1 do.

2017-07-11 Thread Graham Wallis (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082381#comment-16082381
 ] 

Graham Wallis commented on ATLAS-1874:
--

[~ayubkhan] [~ssainath] Thanks for the explanation. I tried running some DSL 
queries (from the UI) and was not getting any results from any queries that 
included any 'select' of an attribute or count.
For example, using the sample data generated by quick_start.py:
Table where db.name="Reporting"   - works correctly, 
returns two tables that belong to the Reporting DB
Table where db.name="Reporting" select owner  - returns nothing
So I think the problem may be broader than just affecting 'count'

> V2 DSL search query does not support for count() but v1 do.
> ---
>
> Key: ATLAS-1874
> URL: https://issues.apache.org/jira/browse/ATLAS-1874
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8.1-incubating
>Reporter: Ayub Pathan
> Fix For: 0.9-incubating, 0.8.1-incubating
>
>
> v1 DSL query supports count()
> {noformat}
> curl -u admin:admin 
> 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/discovery/search/dsl?query=hive_table%20where%20db.name%3D%22default%22%20select%20count()%20as%20%27count%27'
>  | python -m json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100   5370   5370 0339  0 --:--:--  0:00:01 --:--:--   339
> {
> "count": 1,
> "dataType": {
> "attributeDefinitions": [
> {
> "dataTypeName": "long",
> "isComposite": false,
> "isIndexable": false,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "count",
> "reverseAttributeName": null
> }
> ],
> "typeDescription": null,
> "typeName": "__tempQueryResultStruct286",
> "typeVersion": "1.0"
> },
> "query": "hive_table where db.name=\"default\" select count() as 'count'",
> "queryType": "dsl",
> "requestId": "pool-2-thread-9 - 47389e3f-bbf1-4209-8e50-8a3235a7e5a9",
> "results": [
> {
> "$typeName$": "__tempQueryResultStruct286",
> "count": 68
> }
> ]
> }
> {noformat}
> v2 DSL search query does not
> {noformat}
> curl -u admin:admin 
> 'http://ctr-e133-1493418528701-113468-01-02.hwx.site:21000/api/atlas/v2/search/dsl?limit=25=true=where+db.name%3D%22default%22+select+count()+as+%27count%27=hive_table&_=1497434602692'
>  | python -m json.tool
> {
> "queryText": "`hive_table` where db.name=\"default\" select count() as 
> 'count'",
> "queryType": "DSL"
> }
> {noformat}
> *From the initial analysis, it seems like, V2 API response does not have the 
> count attribute which is cause for this failure.* might want to consider 
> adding the count in V2?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1941) HBaseStoreManager is copying host list from root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect zookeeper host

2017-07-11 Thread Ayub Pathan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ayub Pathan updated ATLAS-1941:
---
Summary: HBaseStoreManager is copying host list from root.storage.hostname 
to hbase.zookeeper.quorum, which has one incorrect zookeeper host  (was: 
HBaseStoreManager is overwriting the hbase.zookeeper.quorum value to incorrect 
one)

> HBaseStoreManager is copying host list from root.storage.hostname to 
> hbase.zookeeper.quorum, which has one incorrect zookeeper host
> ---
>
> Key: ATLAS-1941
> URL: https://issues.apache.org/jira/browse/ATLAS-1941
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Priority: Minor
> Fix For: 0.9-incubating
>
>
> While Atlas startup, HBaseStoreManager is copying host list from 
> root.storage.hostname to hbase.zookeeper.quorum, which has one incorrect 
> zookeeper host as well.
> {noformat}
> 2017-07-11 12:13:51,341 INFO  - [main:] ~ Not running setup per configuration 
> atlas.server.run.setup.on.start. (SetupSteps:189)
> 2017-07-11 12:13:52,332 INFO  - [main:] ~ JSR-330 'javax.inject.Inject' 
> annotation found and supported for autowiring 
> (AutowiredAnnotationBeanPostProcessor:155)
> 2017-07-11 12:13:53,038 INFO  - [main:] ~ Instantiated HBase compatibility 
> layer supporting runtime HBase version 1.1.2.2.6.2.0-71: 
> com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 
> (HBaseCompatLoader:69)
> 2017-07-11 12:13:53,108 INFO  - [main:] ~ Copied host list from 
> root.storage.hostname to hbase.zookeeper.quorum: 
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-04.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site
>  (HBaseStoreManager:320)
> {noformat}
> In the above log, the correct zookeeper hosts are 
> ctr-e133-1493418528701-179285-01-05.hwx.site,ctr-e133-1493418528701-179285-01-02.hwx.site.
>  But, as we can see HBaseStoreManager is adding 
> ctr-e133-1493418528701-179285-01-04.hwx.site as well which is incorrect.
> As an outcome of this, zookeeper connection to this host fails.
> {noformat}
> 2017-07-11 12:13:53,399 INFO  - 
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ 
> Opening socket connection to server 
> ctr-e133-1493418528701-179285-01-04.hwx.site/172.27.9.2:2181. Will 
> attempt to SASL-authenticate using Login Context section 'Client' 
> (ClientCnxn:1019)
> 2017-07-11 12:13:53,412 WARN  - 
> [main-SendThread(ctr-e133-1493418528701-179285-01-04.hwx.site:2181):] ~ 
> Session 0x0 for server null, unexpected error, closing socket connection and 
> attempting reconnect (ClientCnxn:1146)
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at 
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
> at 
> org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
> at 
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-11 Thread Sharmadha Sainath (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sharmadha Sainath updated ATLAS-1939:
-
Attachment: Import_error_NPE2.txt

> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: Import_error_NPE2.txt, Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-11 Thread Sharmadha Sainath (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082337#comment-16082337
 ] 

Sharmadha Sainath commented on ATLAS-1939:
--

[~ashutoshm]
I am trying basic import without any option.
{code}
curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F data=@t5.zip 
"http://localhost:21000/api/atlas/admin/import;
{code}

Still it fails with NPE.[^Import_error_NPE2.txt]

But if used with options , import works fine

{code}
curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F data=@t5.zip 
"http://localhost:21000/api/atlas/admin/import; -F request=@t5transform.json
{code}
  

> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: Import_error_NPE2.txt, Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 58422: Addressed Davids review comments

2017-07-11 Thread David Radley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58422/#review180192
---


Ship it!




Ship It!

- David Radley


On July 11, 2017, 2:02 p.m., Graham Wallis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58422/
> ---
> 
> (Updated July 11, 2017, 2:02 p.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> On Windows there is no SIGKILL in the python signal library, so attempting to 
> import SIGKILL causes a fatal error.
> This change introduces a platform switch that avoids trying to import SIGKILL 
> and if the 30sec timeout expires
> it avoids attempting to use SIGKILL, instead using SIGTERM as the os.kill() 
> function on Windows will terminate
> the process and should be as severe as a SIGKILL (kill -9) on a *nix system.
> 
> 
> Diffs
> -
> 
>   distro/src/bin/atlas_stop.py a25d25aee599e7cc9ca3caaff8ff7f11b7e0c789 
> 
> 
> Diff: https://reviews.apache.org/r/58422/diff/3/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Graham Wallis
> 
>

[jira] [Comment Edited] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-11 Thread Ashutosh Mestry (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082249#comment-16082249
 ] 

Ashutosh Mestry edited comment on ATLAS-1939 at 7/11/17 2:26 PM:
-

[~ssainath] The Import API now accepts multi-part data. Please see the updated 
documentation.

This is what will work:
{code:java}
curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F request=@importOptions.json -F 
data=@salesNewTypeAttrs.zip "http://localhost:21000/api/atlas/admin/import;
{code}



was (Author: ashutoshm):
[~ssainath] The Import API now accepts multi-part data. Please see the updated 
documentation.

This is what will work:
{code:java}
curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F data=@salesNewTypeAttrs.zip 
"http://localhost:21000/api/atlas/admin/import;
{code}


> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1939) Export/Import Regression : NPE during import

2017-07-11 Thread Ashutosh Mestry (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082249#comment-16082249
 ] 

Ashutosh Mestry commented on ATLAS-1939:


[~ssainath] The Import API now accepts multi-part data. Please see the updated 
documentation.

This is what will work:
{code:java}
curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F data=@salesNewTypeAttrs.zip 
"http://localhost:21000/api/atlas/admin/import;
{code}


> Export/Import Regression : NPE during import
> 
>
> Key: ATLAS-1939
> URL: https://issues.apache.org/jira/browse/ATLAS-1939
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Blocker
> Attachments: Import_NPE.txt
>
>
> Exported a hive_table and created zip file t5.zip and tried to import in into 
> another cluster using
> {code}
> curl -v -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" --data-binary @t5.zip 
> "http://host2:21000/api/atlas/admin/import;
> {code}
> Import Request failed with 500 internal server error with NPE in application 
> logs.
> Attached the exception stack trace.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1940) Atlas server fails to initialize types on startup from graph store due to duplicate type registered.

2017-07-11 Thread Nixon Rodrigues (JIRA) 
Nixon Rodrigues created ATLAS-1940:
--

 Summary: Atlas server fails to initialize types on startup from 
graph store due to duplicate type registered.
 Key: ATLAS-1940
 URL: https://issues.apache.org/jira/browse/ATLAS-1940
 Project: Atlas
  Issue Type: Bug
  Components: atlas-intg
Affects Versions: 0.8-incubating
Reporter: Nixon Rodrigues
Assignee: Nixon Rodrigues
Priority: Blocker
 Fix For: 0.9-incubating, 0.8.1-incubating


Types are not getting restored from typesRegistry during Atlas startup due 
duplicate type issue.

Below is exception trace.

{code}
2017-07-07 13:35:23,088 INFO  - [main:] ~ Restoring type CLASS.Process.null 
(GraphBackedTypeStore:267)
2017-07-07 13:35:23,099 INFO  - [main:] ~ Restoring type 
CLASS.falcon_process.null (GraphBackedTypeStore:267)
2017-07-07 13:35:23,122 INFO  - [main:] ~ Restoring type TRAIT.Metric.Metric 
Trait (GraphBackedTypeStore:267)
2017-07-07 13:35:23,132 INFO  - [main:] ~ Restoring type 
CLASS.Referenceable.null (GraphBackedTypeStore:267)
2017-07-07 13:35:23,140 INFO  - [main:] ~ Restoring type 
CLASS.hive_storagedesc.null (GraphBackedTypeStore:267)
2017-07-07 13:35:23,203 INFO  - [main:] ~ Restoring type CLASS.hive_table.null 
(GraphBackedTypeStore:267)
2017-07-07 13:35:23,276 INFO  - [main:] ~ Restoring type 
CLASS.falcon_feed_creation.null (GraphBackedTypeStore:267)
2017-07-07 13:35:23,288 INFO  - [main:] ~ Restoring type 
CLASS.sqoop_dbdatastore.null (GraphBackedTypeStore:267)
2017-07-07 13:35:23,399 INFO  - [main:] ~ Number of types got from transient 
type system: 49 (DefaultMetadataService:168)
2017-07-07 13:35:23,406 INFO  - [main:] ~ Restored type system from the store 
(DefaultMetadataService:161)
2017-07-07 13:35:23,407 INFO  - [main:] ~ Global indexes already exist for 
graph (GraphBackedSearchIndexer:141)
2017-07-07 13:35:23,753 WARN  - [main:] ~ Invalid constraint ownedRef for 
attribute hive_storagedesc.serdeInfo (AtlasTypeDefGraphStore:689)
2017-07-07 13:35:23,754 WARN  - [main:] ~ Invalid constraint ownedRef for 
attribute hive_table.aliases (AtlasTypeDefGraphStore:689)
2017-07-07 13:35:23,759 ERROR - [main:] ~ failed to initialize types from graph 
store (AtlasTypeDefGraphStoreV1:77)
org.apache.atlas.exception.AtlasBaseException: Given type TaxonomyTerm already 
exists
at 
org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypeWithNoRefResolve(AtlasTypeRegistry.java:573)
at 
org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypesWithNoRefResolve(AtlasTypeRegistry.java:611)
at 
org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypes(AtlasTypeRegistry.java:407)
at 
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.init(AtlasTypeDefGraphStore.java:105)
at 
org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.init(AtlasTypeDefGraphStoreV1.java:107)
at 
org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1.(AtlasTypeDefGraphStoreV1.java:75)
at 
org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be.()
at 
org.apache.atlas.repository.store.graph.v1.AtlasTypeDefGraphStoreV1$$EnhancerByGuice$$21ccc5be$$FastClassByGuice$$a11fb32d.newInstance()
at 
com.google.inject.internal.ProxyFactory$ProxyConstructor.newInstance(ProxyFactory.java:265)
at 
com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:111)
at 
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:90)
at 
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:268)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:56)
at 
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at 
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
at 
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at 
com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:194)
at 
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41)
at 
com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:205)
at 
com.google.inject.internal.InternalInjectorCreator$1.call(InternalInjectorCreator.java:199)
at 
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1085)
at 
com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:199)
at 
com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:180)
at

Re: Review Request 58422: Addressed Davids review comments

2017-07-11 Thread Graham Wallis 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58422/
---

(Updated July 11, 2017, 2:02 p.m.)


Review request for atlas and David Radley.


Repository: atlas


Description
---

On Windows there is no SIGKILL in the python signal library, so attempting to 
import SIGKILL causes a fatal error.
This change introduces a platform switch that avoids trying to import SIGKILL 
and if the 30sec timeout expires
it avoids attempting to use SIGKILL, instead using SIGTERM as the os.kill() 
function on Windows will terminate
the process and should be as severe as a SIGKILL (kill -9) on a *nix system.


Diffs (updated)
-

  distro/src/bin/atlas_stop.py a25d25aee599e7cc9ca3caaff8ff7f11b7e0c789 


Diff: https://reviews.apache.org/r/58422/diff/3/

Changes: https://reviews.apache.org/r/58422/diff/2-3/


Testing
---

Manually tested


Thanks,

Graham Wallis

[jira] [Updated] (ATLAS-1902) UI: Allow user to add more columns in search table

2017-07-11 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1902:
---
Fix Version/s: 0.9-incubating

> UI: Allow user to add more columns in search table
> --
>
> Key: ATLAS-1902
> URL: https://issues.apache.org/jira/browse/ATLAS-1902
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.9-incubating
>
> Attachments: ATLAS-1902.patch
>
>
> In basic search after searching for a type, user can add more columns based 
> on type selected



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1902) UI: Allow user to add more column in search table

2017-07-11 Thread Keval Bhatt (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1902:
---
Description: In basic search after searching for type user can add more 
column based on type selection. 

> UI: Allow user to add more column in search table
> -
>
> Key: ATLAS-1902
> URL: https://issues.apache.org/jira/browse/ATLAS-1902
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>
> In basic search after searching for type user can add more column based on 
> type selection. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

2017-07-11 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review180183
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 11, 2017, 7:40 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> ---
> 
> (Updated July 11, 2017, 7:40 a.m.)
> 
> 
> Review request for atlas, David Radley and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Create relationshipDefs for all out of the box models. Use relationshipDef 
> information when creating edges for composite attributes in entities.
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 7f64d85e 
>   addons/models/0030-hive_model.json b359ea82 
>   addons/models/0050-falcon_model.json b7398ef4 
>   addons/models/0060-hbase_model.json 42fe00c4 
>   addons/models/0080-storm_model.json 095936e0 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> 0e277b19 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> 8d2e7ec5 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 
> 7c8ae2d7 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> 49a94931 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
>   intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 329dd7a4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
>  4530f51b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  7b3f1e6e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  86212338 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  ebf6a201 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  a5b5730f 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60578/diff/6/
> 
> 
> Testing
> ---
> 
> mvn clean package succeeded. Validated hive_table creation and lineage 
> rendering.
> 
> UTs/ITs in progress.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

[jira] [Reopened] (ATLAS-1733) No SIGKILL in python signal library on Windows

2017-07-11 Thread Graham Wallis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Graham Wallis reopened ATLAS-1733:
--

I think this issue was closed as fixed prematurely. Waiting for review 58422 
completion and will then take advice on how to mark this issue so that the 
change (when acceptable) gets committed.

> No SIGKILL in python signal library on Windows
> --
>
> Key: ATLAS-1733
> URL: https://issues.apache.org/jira/browse/ATLAS-1733
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
> Environment: Windows 7
>Reporter: Graham Wallis
>Assignee: Graham Wallis
>Priority: Minor
> Attachments: ATLAS-1733.patch, ATLAS-1733.patch
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> [I am new to Atlas and this is my first JIRA issue so apologies if I have 
> omitted any necessary information or steps.]
> There is no SIGKILL signal in the python signal library on Windows, so the 
> import in atlas_stop.py fails.
> There are two ways I can suggest that we fix it:
> 1) introduce platform-specific code into atlas_stop.py
> 2) wrap the python os.kill function
> My preference is for the former as it is a small and localized change. 
> In either case the use of SIGTERM should be sufficient on Windows, as the 
> existing python os.kill() function will open the process and terminate it, 
> which is supposed to be equivalent to a SIGKILL (-9) on Linux.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 60681: ATLAS-1698 Glossary OMAS API - early Swagger draft

2017-07-11 Thread Graham Wallis 
Hi David

I have only quickly looked at the glossary OMAS changes but I was 
intrigued by the PList (for pagination). I couldn't see where it is made 
use of. Does something use the PList to access List that it wraps? I had 
expected there to be a getNextPage() method on the PList.

Best regards,
  Graham

Graham Wallis
IBM Analytics Emerging Technology Center
Internet: graham_wal...@uk.ibm.com 
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44-1962-815356Tie: 7-245356




From:   David Radley 
To: Madhan Neethiraj 
Cc: David Radley , Apoorv Naik 
, atlas 
Date:   10/07/2017 22:54
Subject:Re: Review Request 60681: ATLAS-1698 Glossary OMAS API - 
early Swagger draft
Sent by:David Radley 





> On July 9, 2017, 5:13 p.m., Apoorv Naik wrote:
> > webapp/src/main/java/org/apache/atlas/web/rest/GlossaryOMASREST.java
> > Lines 702 (patched)
> > <
https://reviews.apache.org/r/60681/diff/3/?file=1772156#file1772156line702
>
> >
> > Might need to revisit this since HTTP spec allows body with DELETE 
request but the server is expected to ignore the body. (I might be wrong 
too)

Hi Appoorv - thanks for your feedback. For this method it only uses the 
url and a body - have I missed something?


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60681/#review179998
---


On July 10, 2017, 9:31 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60681/
> ---
> 
> (Updated July 10, 2017, 9:31 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is stub code to generate some Glossary OMAS REST interfaces. 
> 
> It does not include : 
> - search
> - finding all assets
> - lineage
> 
> This patch is not to be committed at this time- it is for review 
purposes only.
> 
> 
> Diffs
> -
> 
>   glossaryomas/pom.xml PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/GlossaryOMASCategory.java 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/GlossaryOMASGraphEdgeDirection.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/GlossaryOMASPermission.java 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/AtlasGlossaryGraph.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/AtlasGlossaryGraphEdge.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/AtlasGlossaryGraphNode.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/AtlasGlossaryTreeElement.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/Relatable.java 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/category/AtlasGlossaryCategory.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/glossary/Anchor.java 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/glossary/AtlasGlossary.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/AtlasGlossaryTerm.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/ExternalGlossaryLink.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/Antonym.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/ISA.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/PreferedTerm.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/RelatedTerm.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/RelatedTerms.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/ReplacementTerm.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/Synonym.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/Translation.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/RelatedTerms/ValidValue.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/SemanticAssignment/SemanticAssignment.java
 
PRE-CREATION 
> 
glossaryomas/src/main/java/org/apache/atlas/model/instance/term/TermRelationshipStatus.java
 
PRE-CREATION 
>

[jira] [Updated] (ATLAS-1733) No SIGKILL in python signal library on Windows

2017-07-11 Thread Graham Wallis (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Graham Wallis updated ATLAS-1733:
-
Attachment: ATLAS-1733.patch

Updated to remove trailing whitespace left over from original version of script

> No SIGKILL in python signal library on Windows
> --
>
> Key: ATLAS-1733
> URL: https://issues.apache.org/jira/browse/ATLAS-1733
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
> Environment: Windows 7
>Reporter: Graham Wallis
>Assignee: Graham Wallis
>Priority: Minor
> Attachments: ATLAS-1733.patch, ATLAS-1733.patch
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> [I am new to Atlas and this is my first JIRA issue so apologies if I have 
> omitted any necessary information or steps.]
> There is no SIGKILL signal in the python signal library on Windows, so the 
> import in atlas_stop.py fails.
> There are two ways I can suggest that we fix it:
> 1) introduce platform-specific code into atlas_stop.py
> 2) wrap the python os.kill function
> My preference is for the former as it is a small and localized change. 
> In either case the use of SIGTERM should be sufficient on Windows, as the 
> existing python os.kill() function will open the process and terminate it, 
> which is supposed to be equivalent to a SIGKILL (-9) on Linux.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 58422: Addressed Davids review comments

2017-07-11 Thread Graham Wallis 


> On June 27, 2017, 9:11 a.m., David Radley wrote:
> >

Please ignore the 60747 review (created by mistake). This review (58422) is the 
only one relevant to ATLAS-1733.


> On June 27, 2017, 9:11 a.m., David Radley wrote:
> > distro/src/bin/atlas_stop.py
> > Lines 82 (patched)
> > 
> >
> > formatting error introduced in the fix

I rebased and rebuilt the patch - new review is 
https://reviews.apache.org/r/60747/


- Graham


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58422/#review178961
---


On May 4, 2017, 10:52 a.m., Graham Wallis wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58422/
> ---
> 
> (Updated May 4, 2017, 10:52 a.m.)
> 
> 
> Review request for atlas and David Radley.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> On Windows there is no SIGKILL in the python signal library, so attempting to 
> import SIGKILL causes a fatal error.
> This change introduces a platform switch that avoids trying to import SIGKILL 
> and if the 30sec timeout expires
> it avoids attempting to use SIGKILL, instead using SIGTERM as the os.kill() 
> function on Windows will terminate
> the process and should be as severe as a SIGKILL (kill -9) on a *nix system.
> 
> 
> Diffs
> -
> 
>   distro/src/bin/atlas_stop.py a25d25aee599e7cc9ca3caaff8ff7f11b7e0c789 
> 
> 
> Diff: https://reviews.apache.org/r/58422/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Graham Wallis
> 
>