[jira] [Commented] (ATLAS-2969) Attempt to import empty zip file should not throw excpetion

2018-11-15 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689118#comment-16689118
 ] 

Sarath Subramanian commented on ATLAS-2969:
---

Thanks for the review [~madhan.neethiraj]

> Attempt to import empty zip file should not throw excpetion
> ---
>
> Key: ATLAS-2969
> URL: https://issues.apache.org/jira/browse/ATLAS-2969
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8.3, 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.4, 1.2.0
>
> Attachments: 
> 0001-ATLAS-2969-Attempt-to-import-empty-zip-file-should-n.patch
>
>
> When empty zip files is imported do not throw exception., only log the 
> exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2957) Fix and enable disabled StormHook Integrations Tests

2018-11-06 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2957:
--
Fix Version/s: trunk

> Fix and enable disabled StormHook Integrations Tests
> 
>
> Key: ATLAS-2957
> URL: https://issues.apache.org/jira/browse/ATLAS-2957
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.2.0, trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.2.0, trunk
>
>
> Fix and enable disabled StormHook Integrations Tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2957) Fix and enable disabled StormHook Integrations Tests

2018-11-06 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2957:
-

 Summary: Fix and enable disabled StormHook Integrations Tests
 Key: ATLAS-2957
 URL: https://issues.apache.org/jira/browse/ATLAS-2957
 Project: Atlas
  Issue Type: Task
  Components:  atlas-core
Affects Versions: 1.2.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.2.0


Fix and enable disabled StormHook Integrations Tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2957) Fix and enable disabled StormHook Integrations Tests

2018-11-06 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677414#comment-16677414
 ] 

Sarath Subramanian commented on ATLAS-2957:
---

Code Review: https://reviews.apache.org/r/69264/

> Fix and enable disabled StormHook Integrations Tests
> 
>
> Key: ATLAS-2957
> URL: https://issues.apache.org/jira/browse/ATLAS-2957
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.2.0, trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.2.0, trunk
>
>
> Fix and enable disabled StormHook Integrations Tests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2897) Import Process: Handle Import of Empty Zip Elegantly

2018-10-04 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16638910#comment-16638910
 ] 

Sarath Subramanian commented on ATLAS-2897:
---

+1 for [^ATLAS-2897-Better-handling-of-empty-zip-files.3.patch]

> Import Process: Handle Import of Empty Zip Elegantly
> 
>
> Key: ATLAS-2897
> URL: https://issues.apache.org/jira/browse/ATLAS-2897
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: ATLAS-2897-Better-handling-of-empty-zip-files.3.patch, 
> ATLAS-2897-Elegant-handling-of-empty-zip-files.patch
>
>
> *Background*
> Existing implementation of _ZipSource_ always assumes that the zip that is 
> being passed has valid data. It starts serializing the data and that is where 
> it encounters serialization error. This case can be handled in a more elegant 
> way.
> *Approach Guidance*
>  * In _ZipSource_ ctor, detect case for empty file, throw error message 
> accordingly.
>  * Handle the case where requested _ZipEntry_ is not found.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2905) Generate lineage information for process entities

2018-10-03 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2905:
--
Attachment: 0001-ATLAS-2905-Generate-lineage-information-for-process.1.patch

0001-ATLAS-2905-Generate-lineage-information-for-process.1-branch-0.8.patch

> Generate lineage information for process entities
> -
>
> Key: ATLAS-2905
> URL: https://issues.apache.org/jira/browse/ATLAS-2905
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.2, 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3, 1.1.0
>
> Attachments: 
> 0001-ATLAS-2905-Generate-lineage-information-for-process.1-branch-0.8.patch, 
> 0001-ATLAS-2905-Generate-lineage-information-for-process.1.patch
>
>
> Currently lineage information is generated only for DataSet entities and its 
> derivatives. Lineage information should also be generated for Process 
> entities and in UI lineage tab should be enabled for any process entities and 
> lineage rendered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2905) Generate lineage information for process entities

2018-10-02 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2905:
--
Affects Version/s: 0.8.2

> Generate lineage information for process entities
> -
>
> Key: ATLAS-2905
> URL: https://issues.apache.org/jira/browse/ATLAS-2905
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.2, 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3, 1.1.0
>
>
> Currently lineage information is generated only for DataSet entities and its 
> derivatives. Lineage information should also be generated for Process 
> entities and in UI lineage tab should be enabled for any process entities and 
> lineage rendered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2905) Generate lineage information for process entities

2018-10-02 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636463#comment-16636463
 ] 

Sarath Subramanian commented on ATLAS-2905:
---

Code Review (branch-0.8): https://reviews.apache.org/r/68907/

> Generate lineage information for process entities
> -
>
> Key: ATLAS-2905
> URL: https://issues.apache.org/jira/browse/ATLAS-2905
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.1.0
>
>
> Currently lineage information is generated only for DataSet entities and its 
> derivatives. Lineage information should also be generated for Process 
> entities and in UI lineage tab should be enabled for any process entities and 
> lineage rendered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2905) Generate lineage information for process entities

2018-10-02 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2905:
--
Fix Version/s: 0.8.3

> Generate lineage information for process entities
> -
>
> Key: ATLAS-2905
> URL: https://issues.apache.org/jira/browse/ATLAS-2905
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.2, 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3, 1.1.0
>
>
> Currently lineage information is generated only for DataSet entities and its 
> derivatives. Lineage information should also be generated for Process 
> entities and in UI lineage tab should be enabled for any process entities and 
> lineage rendered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2918) Update JanusGraph version to 0.3.1

2018-10-10 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2918:
-

 Summary: Update JanusGraph version to 0.3.1
 Key: ATLAS-2918
 URL: https://issues.apache.org/jira/browse/ATLAS-2918
 Project: Atlas
  Issue Type: Task
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.0.0


Janusgraph community has released their latest version:

[https://github.com/JanusGraph/janusgraph/releases/tag/v0.3.1]

This JIRA will update janus version to 0.3.1 in atlas source.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2905) Generate lineage information for process entities

2018-10-01 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2905:
-

 Summary: Generate lineage information for process entities
 Key: ATLAS-2905
 URL: https://issues.apache.org/jira/browse/ATLAS-2905
 Project: Atlas
  Issue Type: Task
  Components:  atlas-core
Affects Versions: 1.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.1.0


Currently lineage information is generated only for DataSet entities and its 
derivatives. Lineage information should also be generated for Process entities 
and in UI lineage tab should be enabled for any process entities and lineage 
rendered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2905) Generate lineage information for process entities

2018-10-01 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635055#comment-16635055
 ] 

Sarath Subramanian commented on ATLAS-2905:
---

Code Review: https://reviews.apache.org/r/68893/

> Generate lineage information for process entities
> -
>
> Key: ATLAS-2905
> URL: https://issues.apache.org/jira/browse/ATLAS-2905
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.1.0
>
>
> Currently lineage information is generated only for DataSet entities and its 
> derivatives. Lineage information should also be generated for Process 
> entities and in UI lineage tab should be enabled for any process entities and 
> lineage rendered.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2906) Import Process: Transforms: Handling Custom Attributes Does Not Update

2018-10-03 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637770#comment-16637770
 ] 

Sarath Subramanian commented on ATLAS-2906:
---

+1

> Import Process: Transforms: Handling Custom Attributes Does Not Update
> --
>
> Key: ATLAS-2906
> URL: https://issues.apache.org/jira/browse/ATLAS-2906
> Project: Atlas
>  Issue Type: Bug
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: 
> ATLAS-2906-Allow-transforms-to-be-applied-when-entit.patch
>
>
> *Background*
> The new transform framework looks at the entity holistically and computes 
> attributes as it sees fit.
> Updating the _hive_storagedesc.location_ if entity-level transform is 
> present, does not apply the transform.
> *Steps* *to Duplicate*
>  * Attempt to import a zip file with the following transform.
> {code:java}
> { "options": { "transformers": "[{\"conditions\":{\"__entity\":\"topLevel: 
> \"},\"action\":{\"__entity\":\"ADD_CLASSIFICATION: 
> mycluster0_replicated\"}},{\"action\":{\"__entity.replicatedTo\":\"CLEAR:\",\"__entity.replicatedFrom\":\"CLEAR:\"}},{\"conditions\":{\"hive_db.clusterName\":\"EQUALS:
>  mycluster0\"},\"action\":{\"hive_db.clusterName\":\"SET: 
> mycluster1\"}},{\"conditions\":{\"hive_storagedesc.location\":\"STARTS_WITH_IGNORE_CASE:
>  
> hdfs://localhost.localdomain\"},\"action\":{\"hive_storagedesc.location\":\"REPLACE_PREFIX:
>  = :hdfs://localhost.localdomain=hdfs://mycluster1\"}}]", "replicatedFrom": 
> "SFO$clx" } }
> {code}
> Locate any entity of type _hive_storagedesc_ and note the _location._ It does 
> not apply the updated location.
> *Root Cause*
>  * The new transform framework re-computes the location based on cached data.
>  * Solution: Prevent caching of this attribute so that it honors the 
> externally set transformation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2885) Skip handling of INSERT INTO...VALUES hive query messages

2018-09-21 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2885:
--
Attachment: 0001-ATLAS-2885-Skip-handling-of-INSERT-INTO.VALUES-hive-.patch

> Skip handling of INSERT INTO...VALUES hive query messages
> -
>
> Key: ATLAS-2885
> URL: https://issues.apache.org/jira/browse/ATLAS-2885
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.2
>Reporter: Sarath Subramanian
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 0.8.3
>
> Attachments: 
> 0001-ATLAS-2885-Skip-handling-of-INSERT-INTO.VALUES-hive-.patch
>
>
> The INSERT INTO...VALUES() statement used by hive to insert data into hive 
> tables generates lineage in atlas for every INSERT INTO..VALUES statement.
> The lineage generated is something like:
> *Temp_Table -> HiveTable*
> This is a transient table and is not persisted. Atlas should skip handling of 
> this lineage.
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2892) TypeREST: Provide REST Endpoint that Deletes Type Given Type Name

2018-09-25 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628025#comment-16628025
 ] 

Sarath Subramanian commented on ATLAS-2892:
---

Review comments:
 # Please review incorrect java doc params mentioned
 # Handle case if typename is not found in store 
 # In master/branch-1.0 - AtlasRelationshipDef needs to be handled.
 # Can we use typeregistry to get the AtlasBaseTypeDef insteading of checking 
the store?

> TypeREST: Provide REST Endpoint that Deletes Type Given Type Name
> -
>
> Key: ATLAS-2892
> URL: https://issues.apache.org/jira/browse/ATLAS-2892
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.8.2, trunk
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: 0.8.2, trunk
>
> Attachments: 
> ATLAS-2892-Additional-TypesREST-endpoint-for-deletin.patch
>
>
> *Background*
> Existing Type REST API for deleting type takes _AtlasTypesDef_ as parameter. 
> This is useful for bulk delete. For single type delete it will be helpful to 
> have a REST endpoint that performs type delete by taking name of the time.
> *Approach Guideline*
>  * Update _TypesREST_.
>  * Add new method.
>  * Continue to use _typesdefStore.deleteTypesDef_



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2885) Skip handling of INSERT INTO...VALUES hive query messages

2018-09-25 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628264#comment-16628264
 ] 

Sarath Subramanian commented on ATLAS-2885:
---

Thanks for the patch [~apoorvnaik]. Committed.

> Skip handling of INSERT INTO...VALUES hive query messages
> -
>
> Key: ATLAS-2885
> URL: https://issues.apache.org/jira/browse/ATLAS-2885
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.2
>Reporter: Sarath Subramanian
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 0.8.3
>
> Attachments: 
> 0001-ATLAS-2885-Skip-handling-of-INSERT-INTO.VALUES-hive-.patch
>
>
> The INSERT INTO...VALUES() statement used by hive to insert data into hive 
> tables generates lineage in atlas for every INSERT INTO..VALUES statement.
> The lineage generated is something like:
> *Temp_Table -> HiveTable*
> This is a transient table and is not persisted. Atlas should skip handling of 
> this lineage.
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2890) Fix intermittent UT and IT failures for atlas in apache CI

2018-09-25 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2890:
-

 Summary: Fix intermittent UT and IT failures for atlas in apache CI
 Key: ATLAS-2890
 URL: https://issues.apache.org/jira/browse/ATLAS-2890
 Project: Atlas
  Issue Type: Task
  Components:  atlas-core
Affects Versions: 1.0.0, 0.8.2
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 0.8.3, 1.1.0


We see intermittent test failures in 
[https://builds.apache.org/view/A/view/Atlas/]

This Jira addresses fixes for these intermittent failures in branch-0.8, 
branch-1.0 and master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2890) Fix intermittent UT and IT failures for atlas in apache CI

2018-09-25 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2890:
--
Attachment: 0001-ATLAS-2890-Fix-intermittent-UT-and-IT-failures-for-a.patch

> Fix intermittent UT and IT failures for atlas in apache CI
> --
>
> Key: ATLAS-2890
> URL: https://issues.apache.org/jira/browse/ATLAS-2890
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.2, 1.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3, 1.1.0
>
> Attachments: 
> 0001-ATLAS-2890-Fix-intermittent-UT-and-IT-failures-for-a.patch
>
>
> We see intermittent test failures in 
> [https://builds.apache.org/view/A/view/Atlas/]
> This Jira addresses fixes for these intermittent failures in branch-0.8, 
> branch-1.0 and master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2971) Atlas fails to start if there is a empty pid file

2018-11-16 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690022#comment-16690022
 ] 

Sarath Subramanian commented on ATLAS-2971:
---

looks good. +1 for the patch.

> Atlas fails to start if there is a empty pid file
> -
>
> Key: ATLAS-2971
> URL: https://issues.apache.org/jira/browse/ATLAS-2971
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sandeep Nemuri
>Priority: Major
> Attachments: ATLAS-2971.001.patch
>
>
> Atlas fails to start with below error if the pid file is empty.
>  
> {code:java}
> ~]$ touch /var/run/atlas/atlas.pid
> ~]$ source 
> /usr/hdp/current/atlas-server/conf/atlas-env.sh;/usr/hdp/current/atlas-server/bin/atlas_start.py
>  
> Exception: invalid literal for int() with base 10: '' 
> Traceback (most recent call last): 
> File "/usr/hdp/current/atlas-server/bin/atlas_start.py", line 163, in 
>  
> returncode = main() 
> File "/usr/hdp/current/atlas-server/bin/atlas_start.py", line 105, in main 
> if mc.exist_pid((int)(pid)): 
> ValueError: invalid literal for int() with base 10: '' 
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3034) Perf enhancement to avoid unnecessary lookup when creating new relationships

2019-01-21 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748271#comment-16748271
 ] 

Sarath Subramanian commented on ATLAS-3034:
---

Perf improvement - creating hive_table with 5000 columns

*Before patch:*
2019-01-21 19:27:17,223|NotificationHookConsumer 
thread-0|PERF|createOrUpdate()|1847974
2019-01-21 19:28:12,778|NotificationHookConsumer 
thread-0|PERF|ENTITY_CREATE_V2|1903529 (*~ 31 minutes*)

*After patch:*
2019-01-21 20:05:19,227|NotificationHookConsumer 
thread-0|PERF|createOrUpdate()|450430
2019-01-21 20:06:14,423|NotificationHookConsumer 
thread-0|PERF|ENTITY_CREATE_V2|505626 (*~ 8.4 minutes*)

> Perf enhancement to avoid unnecessary lookup when creating new relationships
> 
>
> Key: ATLAS-3034
> URL: https://issues.apache.org/jira/browse/ATLAS-3034
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
>
> Perf enhancement to avoid unnecessary lookup when creating new relationships



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3034) Perf enhancement to avoid unnecessary lookup when creating new relationships

2019-01-21 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3034:
-

 Summary: Perf enhancement to avoid unnecessary lookup when 
creating new relationships
 Key: ATLAS-3034
 URL: https://issues.apache.org/jira/browse/ATLAS-3034
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 1.2.0, 2.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.2.0, 2.0.0


Perf enhancement to avoid unnecessary lookup when creating new relationships



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3034) Perf enhancement to avoid unnecessary lookup when creating new relationships

2019-01-21 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748270#comment-16748270
 ] 

Sarath Subramanian commented on ATLAS-3034:
---

Code review: https://reviews.apache.org/r/69804/

> Perf enhancement to avoid unnecessary lookup when creating new relationships
> 
>
> Key: ATLAS-3034
> URL: https://issues.apache.org/jira/browse/ATLAS-3034
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
>
> Perf enhancement to avoid unnecessary lookup when creating new relationships



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (ATLAS-3034) Perf enhancement to avoid unnecessary lookup when creating new relationships

2019-01-22 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748271#comment-16748271
 ] 

Sarath Subramanian edited comment on ATLAS-3034 at 1/23/19 1:28 AM:


Perf improvement - creating hive_table with 5000 columns

*Before patch:*
 2019-01-21 19:27:17,223|NotificationHookConsumer 
thread-0|PERF|createOrUpdate()|1847974
 2019-01-21 19:28:12,778|NotificationHookConsumer 
thread-0|PERF|ENTITY_CREATE_V2|1903529 (*~ 31 minutes*)

*After patch:*
 2019-01-21 20:05:19,227|NotificationHookConsumer 
thread-0|PERF|createOrUpdate()|450430
 2019-01-21 20:06:14,423|NotificationHookConsumer 
thread-0|PERF|ENTITY_CREATE_V2|505626 (*~ 6.89 minutes*)


was (Author: sarath.ku...@gmail.com):
Perf improvement - creating hive_table with 5000 columns

*Before patch:*
2019-01-21 19:27:17,223|NotificationHookConsumer 
thread-0|PERF|createOrUpdate()|1847974
2019-01-21 19:28:12,778|NotificationHookConsumer 
thread-0|PERF|ENTITY_CREATE_V2|1903529 (*~ 31 minutes*)

*After patch:*
2019-01-21 20:05:19,227|NotificationHookConsumer 
thread-0|PERF|createOrUpdate()|450430
2019-01-21 20:06:14,423|NotificationHookConsumer 
thread-0|PERF|ENTITY_CREATE_V2|505626 (*~ 8.4 minutes*)

> Perf enhancement to avoid unnecessary lookup when creating new relationships
> 
>
> Key: ATLAS-3034
> URL: https://issues.apache.org/jira/browse/ATLAS-3034
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.2.0, 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
>
> Perf enhancement to avoid unnecessary lookup when creating new relationships



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2969) Attempt to import empty zip file should not throw excpetion

2018-12-10 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715374#comment-16715374
 ] 

Sarath Subramanian commented on ATLAS-2969:
---

Thanks for the patch [~ashutoshm]. +1.

> Attempt to import empty zip file should not throw excpetion
> ---
>
> Key: ATLAS-2969
> URL: https://issues.apache.org/jira/browse/ATLAS-2969
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8.3, 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.4, 1.2.0
>
> Attachments: 
> 0001-ATLAS-2969-Attempt-to-import-empty-zip-file-should-n.patch, 
> ATLAS-2969-Addressed-UniformInterfaceException-from-.patch
>
>
> When empty zip files is imported do not throw exception., only log the 
> exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2987) Update component versions of Atlas to use Hadoop3, HBase2 and Solr7

2018-12-10 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2987:
--
Attachment: 0001-ATLAS-2987-Update-component-versions-of-Atlas-to-use.patch

> Update component versions of Atlas to use Hadoop3, HBase2 and Solr7
> ---
>
> Key: ATLAS-2987
> URL: https://issues.apache.org/jira/browse/ATLAS-2987
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 
> 0001-ATLAS-2987-Update-component-versions-of-Atlas-to-use.patch
>
>
> Update Component version of Atlas to the following:
>  * Hadoop 3.1.1
>  * Hive 3.1.0
>  * HBase 2.0.2
>  * Solr 7.5.0
>  * Kafka 2.0.0
>  * Zookeeper 3.4.6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2987) Update component versions of Atlas to use Hadoop3, HBase2 and Solr7

2018-12-10 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716110#comment-16716110
 ] 

Sarath Subramanian commented on ATLAS-2987:
---

Code Review: https://reviews.apache.org/r/69546/

> Update component versions of Atlas to use Hadoop3, HBase2 and Solr7
> ---
>
> Key: ATLAS-2987
> URL: https://issues.apache.org/jira/browse/ATLAS-2987
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 
> 0001-ATLAS-2987-Update-component-versions-of-Atlas-to-use.patch
>
>
> Update Component version of Atlas to the following:
>  * Hadoop 3.1.1
>  * Hive 3.1.0
>  * HBase 2.0.2
>  * Solr 7.5.0
>  * Kafka 2.0.0
>  * Zookeeper 3.4.6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2987) Update component versions of Atlas to use Hadoop3, HBase2 and Solr6

2018-12-10 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2987:
--
Description: 
Update Component version of Atlas to the following:
 * Hadoop 3.1.1
 * Hive 3.1.0
 * HBase 2.0.2
 * Solr 7.5.0
 * Kafka 2.0.0
 * Zookeeper 3.4.6

  was:
Update Component version of Atlas to the following:
 * Hadoop 3.0.0
 * Hive 3.0.0
 * HBase 2.0.0
 * Solr 6.6.1
 * Kafka 1.0.0
 * Zookeeper 3.4.6


> Update component versions of Atlas to use Hadoop3, HBase2 and Solr6
> ---
>
> Key: ATLAS-2987
> URL: https://issues.apache.org/jira/browse/ATLAS-2987
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> Update Component version of Atlas to the following:
>  * Hadoop 3.1.1
>  * Hive 3.1.0
>  * HBase 2.0.2
>  * Solr 7.5.0
>  * Kafka 2.0.0
>  * Zookeeper 3.4.6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2979) Added service type dropdown to filter entitydef type.

2018-12-21 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727002#comment-16727002
 ] 

Sarath Subramanian commented on ATLAS-2979:
---

UI patch for adding service-type filter looks good - ATLAS-2979_FINAL.patch. +1

> Added service type dropdown to filter entitydef type.
> -
>
> Key: ATLAS-2979
> URL: https://issues.apache.org/jira/browse/ATLAS-2979
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>Affects Versions: 2.0.0
>Reporter: Abhishek Kadam
>Assignee: Abhishek Kadam
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-2979.patch, ATLAS-2979_1.patch, 
> ATLAS-2979_FINAL.patch, ATLAS-2979_FINAL.patch, 
> ATLAS-2979_UI_Improvement.patch, return-serviceType-in-api-header.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2983) Fixes and clean up for Atlas Kafka bridge utility

2018-11-29 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704116#comment-16704116
 ] 

Sarath Subramanian commented on ATLAS-2983:
---

+1 Thanks for the patch [~umesh.padashetty]

> Fixes and clean up for Atlas Kafka bridge utility
> -
>
> Key: ATLAS-2983
> URL: https://issues.apache.org/jira/browse/ATLAS-2983
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core, atlas-webui
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Umesh Padashetty
>Priority: Major
> Attachments: ATLAS-2983.patch
>
>
> # Currently, import-kafka.sh script allows selective kafka topic import by 
> providing -t or --topic option, but the script only checks for the topics 
> which start with the topic name provided and does not support regular 
> expression based selective import. With the fix provided, regex patterns can 
> be used along with -t as well as -f (file based import) option
>  # When a bad import-kafka command is being fired, following usage is being 
> printed on the console. Here, option should be -t instead of -n. Have fixed 
> this minor issue, along with some more text changes
>  ** import-kafka.sh [*-n*  OR --topic ]
>  # Since we now have support for kakfa bridge and hbase hook/bridge, option 
> for creating entities for the following in Atlas UI could be removed 
>  ## hbase_table
>  ## hbase_column
>  ## hbase_column_family
>  ## kafka_topic
>  ## hbase_namespace



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2987) Update component versions of Atlas to use Hadoop3, HBase2 and Solr6

2018-11-30 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2987:
--
Description: 
Update Component version of Atlas to the following:
 * Hadoop 3.0.0
 * Hive 3.0.0
 * HBase 2.0.0
 * Solr 6.6.1
 * Kafka 1.0.0
 * Zookeeper 3.4.6

  was:
Update Component version of Atlas to the following:
 * Hadoop 3.0.0
 * HBase 2.0.0
 * Solr 6.6.1
 * Kafka 1.0.0
 * Zookeeper 3.4.6
 * Hive 3.0.0


> Update component versions of Atlas to use Hadoop3, HBase2 and Solr6
> ---
>
> Key: ATLAS-2987
> URL: https://issues.apache.org/jira/browse/ATLAS-2987
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> Update Component version of Atlas to the following:
>  * Hadoop 3.0.0
>  * Hive 3.0.0
>  * HBase 2.0.0
>  * Solr 6.6.1
>  * Kafka 1.0.0
>  * Zookeeper 3.4.6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2987) Update component versions of Atlas to use Hadoop3, HBase2 and Solr6

2018-11-30 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2987:
-

 Summary: Update component versions of Atlas to use Hadoop3, HBase2 
and Solr6
 Key: ATLAS-2987
 URL: https://issues.apache.org/jira/browse/ATLAS-2987
 Project: Atlas
  Issue Type: Task
  Components:  atlas-core
Affects Versions: trunk
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


Update Component version of Atlas to the followinf:
 * Hadoop 3.0.0
 * HBase 2.0.0
 * Solr 6.6.1
 * Kafka 1.0.0
 * Zookeeper 3.4.6
 * Hive 3.0.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2987) Update component versions of Atlas to use Hadoop3, HBase2 and Solr6

2018-11-30 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2987:
--
Description: 
Update Component version of Atlas to the following:
 * Hadoop 3.0.0
 * HBase 2.0.0
 * Solr 6.6.1
 * Kafka 1.0.0
 * Zookeeper 3.4.6
 * Hive 3.0.0

  was:
Update Component version of Atlas to the followinf:
 * Hadoop 3.0.0
 * HBase 2.0.0
 * Solr 6.6.1
 * Kafka 1.0.0
 * Zookeeper 3.4.6
 * Hive 3.0.0


> Update component versions of Atlas to use Hadoop3, HBase2 and Solr6
> ---
>
> Key: ATLAS-2987
> URL: https://issues.apache.org/jira/browse/ATLAS-2987
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> Update Component version of Atlas to the following:
>  * Hadoop 3.0.0
>  * HBase 2.0.0
>  * Solr 6.6.1
>  * Kafka 1.0.0
>  * Zookeeper 3.4.6
>  * Hive 3.0.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2976) UI : Show entity specific icon for the lineage node

2018-12-03 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2976:
--
Fix Version/s: 2.0.0
   1.2.0

> UI : Show entity specific icon for the lineage node
> ---
>
> Key: ATLAS-2976
> URL: https://issues.apache.org/jira/browse/ATLAS-2976
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 1.2.0, 2.0.0
>
> Attachments: ATLAS-2976-1.patch, ATLAS-2976-2.patch, 
> ATLAS-2976-3.patch, ATLAS-2976-4.patch, ATLAS-2976.patch, 
> New_Lineage_detailpage_layout.png, New_way_to_select_filter.png, 
> Old_Lineage_detailpage_layout.png
>
>
> How to configure custom image(icon) for the entity type.
>  # Create the image (icon) with the "*.png*"  extension
>  # Add your new image in 
> [../dashboardv2/public/img/entity-icon/|https://github.com/apache/atlas/tree/master/dashboardv2/public/img/entity-icon]
>  /
>  
>  Note: "*If the above path doesn't have the image then default image is 
> placed as entity icon*"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2976) UI : Show entity specific icon for the lineage node

2018-12-03 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707819#comment-16707819
 ] 

Sarath Subramanian commented on ATLAS-2976:
---

+1. Thanks [~kevalbhatt]

> UI : Show entity specific icon for the lineage node
> ---
>
> Key: ATLAS-2976
> URL: https://issues.apache.org/jira/browse/ATLAS-2976
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-2976-1.patch, ATLAS-2976-2.patch, 
> ATLAS-2976-3.patch, ATLAS-2976-4.patch, ATLAS-2976.patch, 
> New_Lineage_detailpage_layout.png, New_way_to_select_filter.png, 
> Old_Lineage_detailpage_layout.png
>
>
> How to configure custom image(icon) for the entity type.
>  # Create the image (icon) with the "*.png*"  extension
>  # Add your new image in 
> [../dashboardv2/public/img/entity-icon/|https://github.com/apache/atlas/tree/master/dashboardv2/public/img/entity-icon]
>  /
>  
>  Note: "*If the above path doesn't have the image then default image is 
> placed as entity icon*"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2970) UI: Show counts besides the Type and Classification dropdown list in search

2018-11-21 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695239#comment-16695239
 ] 

Sarath Subramanian commented on ATLAS-2970:
---

Thanks for the patch [~abhib4]. Looks good and the count shows correctly now. 

> UI: Show counts besides the Type and Classification dropdown list in search
> ---
>
> Key: ATLAS-2970
> URL: https://issues.apache.org/jira/browse/ATLAS-2970
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>Affects Versions: 2.0.0
>Reporter: Abhishek Kadam
>Assignee: Abhishek Kadam
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-2970.patch, ATLAS-2970_1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2979) Added service type dropdown to filter entitydef type.

2018-11-23 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16697531#comment-16697531
 ] 

Sarath Subramanian commented on ATLAS-2979:
---

For *return-serviceType-in-api-header-updated.patch -* Include _service-type_ 
in hashCode()

> Added service type dropdown to filter entitydef type.
> -
>
> Key: ATLAS-2979
> URL: https://issues.apache.org/jira/browse/ATLAS-2979
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>Affects Versions: 2.0.0
>Reporter: Abhishek Kadam
>Assignee: Abhishek Kadam
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-2979.patch, 
> return-serviceType-in-api-header-updated.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3020) ClientV2: DeleteClassifications Invokes Incorrect REST Call

2019-01-03 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733591#comment-16733591
 ] 

Sarath Subramanian commented on ATLAS-3020:
---

+1 for the patch. Thanks [~ashutoshm].

> ClientV2: DeleteClassifications Invokes Incorrect REST Call
> ---
>
> Key: ATLAS-3020
> URL: https://issues.apache.org/jira/browse/ATLAS-3020
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: 1.1.0, 0.8.4, trunk
>
> Attachments: ATLAS-3020-Atlas-Client-Delete-Classifications-API.patch
>
>
> *Background*
> Existing _AtlasClientV2.deleteClassifications_ invokes incorrect REST API. 
> *Solution*
> The method should invoke the correct REST API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2874) Include handling of Atlas Entity Transformers in current Import logic

2018-09-17 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618197#comment-16618197
 ] 

Sarath Subramanian commented on ATLAS-2874:
---

Code Review: https://reviews.apache.org/r/68740/

> Include handling of Atlas Entity Transformers in current Import logic
> -
>
> Key: ATLAS-2874
> URL: https://issues.apache.org/jira/browse/ATLAS-2874
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8.2
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3
>
>
> Include handling of Atlas Entity Transformers implemented in ATLAS-2873 in 
> current Import logic



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2873) Atlas Import Transform Handler Implementation

2018-09-16 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16616871#comment-16616871
 ] 

Sarath Subramanian commented on ATLAS-2873:
---

Code Review: https://reviews.apache.org/r/68713/

> Atlas Import Transform Handler Implementation
> -
>
> Key: ATLAS-2873
> URL: https://issues.apache.org/jira/browse/ATLAS-2873
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.8.3
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3
>
>
> Implement atlas entity transformation during import.
>  
> e.g.
> *Condition Operators :* EQUALS, STARTS_WITH
> *Action Operators*   : SET, TO_LOWER, TO_UPPER, REPLACE_PREFIX
>  
> *Condition :*  "hdfs_path.clusterName", "EQUALS: cl1"
> *Action   :*  "hdfs_path.clusterName", "SET: cl2"
>  
> *Condition :*  "hdfs_path.clusterName", "EQUALS: cl1"
> *Action   :*  " hdfs_path.clusterName", "TO_UPPER:"
>  
> *Condition :*  " hdfs_path.clusterName", "EQUALS: CL1"
> *Action   :*  " hdfs_path.clusterName", "TO_LOWER:" 
>  
> *Condition :*  " hdfs_path.name", "STARTS_WITH: /aa/bb/"
> *Action   :*  " hdfs_path.name", "REPLACE_PREFIX: = :/aa/bb/=/xx/yy/"
>  
> *Condition :*  "hive_db.clusterName", "EQUALS: cl1"
> *Action   :*  "hive_db.clusterName", "SET: cl1_backup; 
>  
> *Condition :*  "hive_db.name", "EQUALS: hr"
> *Action   :*  "hive_db.name", "SET: hr_backup"
>  
> *Condition :*  "hive_table.name", "EQUALS: employees"
> *Action   :*  "hive_table.name", "SET: employees_backup"
>  
> *Condition :*  "hive_column.name", "EQUALS: age"
> *Action   :*  "hive_column.name", "SET: age_backup"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2873) Atlas Import Transform Handler Implementation

2018-09-16 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2873:
-

 Summary: Atlas Import Transform Handler Implementation
 Key: ATLAS-2873
 URL: https://issues.apache.org/jira/browse/ATLAS-2873
 Project: Atlas
  Issue Type: Task
  Components:  atlas-core
Affects Versions: 0.8.3
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 0.8.3


Implement atlas entity transformation during import.

 

e.g.

*Condition Operators :* EQUALS, STARTS_WITH

*Action Operators*   : SET, TO_LOWER, TO_UPPER, REPLACE_PREFIX

 

*Condition :*  "hdfs_path.clusterName", "EQUALS: cl1"

*Action   :*  "hdfs_path.clusterName", "SET: cl2"

 

*Condition :*  "hdfs_path.clusterName", "EQUALS: cl1"

*Action   :*  " hdfs_path.clusterName", "TO_UPPER:"

 

*Condition :*  " hdfs_path.clusterName", "EQUALS: CL1"

*Action   :*  " hdfs_path.clusterName", "TO_LOWER:" 

 

*Condition :*  " hdfs_path.name", "STARTS_WITH: /aa/bb/"

*Action   :*  " hdfs_path.name", "REPLACE_PREFIX: = :/aa/bb/=/xx/yy/"

 

*Condition :*  "hive_db.clusterName", "EQUALS: cl1"
*Action   :*  "hive_db.clusterName", "SET: cl1_backup; 

 

*Condition :*  "hive_db.name", "EQUALS: hr"
*Action   :*  "hive_db.name", "SET: hr_backup"

 

*Condition :*  "hive_table.name", "EQUALS: employees"
*Action   :*  "hive_table.name", "SET: employees_backup"

 

*Condition :*  "hive_column.name", "EQUALS: age"
*Action   :*  "hive_column.name", "SET: age_backup"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2875) Implement clear attribute value transformer for Atlas Entity Transformer

2018-09-17 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2875:
-

 Summary: Implement clear attribute value transformer for Atlas 
Entity Transformer
 Key: ATLAS-2875
 URL: https://issues.apache.org/jira/browse/ATLAS-2875
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 0.8.2
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 0.8.3


*Condition :*  "hive_table.replicatedTo", "HAS_VALUE:"

*Action   :*  "hive_table.replicatedTo", "CLEAR:"

 

*Condition :*  null

*Action   :*  "hive_table.replicatedTo", "CLEAR:"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-2875) Implement clear attribute value transformer for Atlas Entity Transformer

2018-09-17 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618489#comment-16618489
 ] 

Sarath Subramanian commented on ATLAS-2875:
---

Code Review: https://reviews.apache.org/r/68743/

> Implement clear attribute value transformer for Atlas Entity Transformer
> 
>
> Key: ATLAS-2875
> URL: https://issues.apache.org/jira/browse/ATLAS-2875
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8.2
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.3
>
>
> *Condition :*  "hive_table.replicatedTo", "HAS_VALUE:"
> *Action   :*  "hive_table.replicatedTo", "CLEAR:"
>  
> *Condition :*  null
> *Action   :*  "hive_table.replicatedTo", "CLEAR:"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-2781) Hard Delete : Deleting an entity which is associated to an term associated to a tag fails

2018-09-18 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2781:
--
Fix Version/s: (was: 1.1.0)
   1.2.0

> Hard Delete : Deleting an entity which is associated to an term associated to 
> a tag fails
> -
>
> Key: ATLAS-2781
> URL: https://issues.apache.org/jira/browse/ATLAS-2781
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Critical
> Fix For: 1.2.0
>
> Attachments: 
> 0001-ATLAS-2781-Hard-Delete-Deleting-an-entity-which-is-a.patch
>
>
> 1. Create a term term1 under glossary glossary1 .
> 2. Create tag tag1.
> 3. Associate tag1 to term1.
> 4. Create a hive_db db1 .
> 5. Associate db1 to term1.
> 6. Drop db1.
> When hard delete is configured , delete db entity operation fails with 
> {code}
> Max retries exceeded for message [AtlasObjectId\{guid='null', 
> typeName='hive_db', uniqueAttributes={qualifiedName:db12@cl1}}] 
> (NotificationHookConsumer:531)
> org.apache.atlas.exception.AtlasBaseException: Given instance guid 
> 243874d4-9e5a-4411-b170-b73045a59267 is invalid/not found
> {code}
> Attached the complete stack trace.
> Note:
> In the following scenarios , delete works properly:
> 1. Deleting the db without any tag/term association
> 2. Deleting the db with either tag or term association
> 3. Deleting the db associated to a term and also a tag.
> Issue happens only when the entity is associated to a term, which is 
> associated to a tag.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-2874) Include handling of Atlas Entity Transformers in current Import logic

2018-09-17 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-2874:
-

 Summary: Include handling of Atlas Entity Transformers in current 
Import logic
 Key: ATLAS-2874
 URL: https://issues.apache.org/jira/browse/ATLAS-2874
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 0.8.2
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 0.8.3


Include handling of Atlas Entity Transformers implemented in ATLAS-2873 in 
current Import logic



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3036) Improve FullTextMapper performance during entity retrieval

2019-01-24 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3036:
-

 Summary: Improve FullTextMapper performance during entity retrieval
 Key: ATLAS-3036
 URL: https://issues.apache.org/jira/browse/ATLAS-3036
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 0.8.3
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 0.8.4


Improve FullTextMapper performance during entity retrieval



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (ATLAS-3036) Improve FullTextMapper performance during entity retrieval

2019-01-27 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751704#comment-16751704
 ] 

Sarath Subramanian edited comment on ATLAS-3036 at 1/28/19 12:57 AM:
-

Code Review:

master: [https://reviews.apache.org/r/69846/]

branch-0.8: [https://reviews.apache.org/r/69835/]

 

 


was (Author: sarath.ku...@gmail.com):
Code Review: [https://reviews.apache.org/r/69835/]

 

> Improve FullTextMapper performance during entity retrieval
> --
>
> Key: ATLAS-3036
> URL: https://issues.apache.org/jira/browse/ATLAS-3036
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8.3
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 0.8.4
>
> Attachments: 
> 0001-ATLAS-3036-Improve-FullTextMapper-performance-during-MASTER.patch, 
> 0001-ATLAS-3036-Improve-FullTextMapper-performance-during.patch
>
>
> Improve FullTextMapper performance during entity retrieval



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3113) Use index query to search for active entities and better logging in java patch framework

2019-04-01 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3113:
-

 Summary: Use index query to search for active entities and better 
logging in java patch framework
 Key: ATLAS-3113
 URL: https://issues.apache.org/jira/browse/ATLAS-3113
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


In java patch framework - Unique attribute patch handler - to retrieve all 
active entities graph query is used, graph query is expensive when number of 
entities are large. This Jira updates the graph query to use index query and 
also logging enhancements.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3112) Allow Indexing of array attributes (LIST or SET) in indexing store

2019-04-01 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3112:
-

 Summary: Allow Indexing of array attributes (LIST or SET) in 
indexing store
 Key: ATLAS-3112
 URL: https://issues.apache.org/jira/browse/ATLAS-3112
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


Array attributes with cardinality LIST or SET is not indexed in the indexing 
store. To allow index query to search on array attributes, array attributes 
needs to be indexed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3113) Use index query to search for active entities and better logging in java patch framework

2019-04-01 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807405#comment-16807405
 ] 

Sarath Subramanian commented on ATLAS-3113:
---

Code review: [https://reviews.apache.org/r/70358/]

> Use index query to search for active entities and better logging in java 
> patch framework
> 
>
> Key: ATLAS-3113
> URL: https://issues.apache.org/jira/browse/ATLAS-3113
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> In java patch framework - Unique attribute patch handler - to retrieve all 
> active entities graph query is used, graph query is expensive when number of 
> entities are large. This Jira updates the graph query to use index query and 
> also logging enhancements.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3112) Allow Indexing of array attributes (LIST or SET) in indexing store

2019-04-01 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16807404#comment-16807404
 ] 

Sarath Subramanian commented on ATLAS-3112:
---

Code review: [https://reviews.apache.org/r/70357/]

> Allow Indexing of array attributes (LIST or SET) in indexing store
> --
>
> Key: ATLAS-3112
> URL: https://issues.apache.org/jira/browse/ATLAS-3112
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> Array attributes with cardinality LIST or SET is not indexed in the indexing 
> store. To allow index query to search on array attributes, array attributes 
> needs to be indexed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3109) Allow ignoring relationshipAttributes for AtlasEntityStore.getByUniqueAttributes

2019-03-31 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806212#comment-16806212
 ] 

Sarath Subramanian commented on ATLAS-3109:
---

+1 for patch ATLAS-3109-2.patch

> Allow ignoring relationshipAttributes for 
> AtlasEntityStore.getByUniqueAttributes
> 
>
> Key: ATLAS-3109
> URL: https://issues.apache.org/jira/browse/ATLAS-3109
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Jiaqi Shan
>Priority: Major
> Attachments: ATLAS-3109-2.patch, 
> ATLAS-3109-Allow-ignoring-relationshipAttributes-for.patch
>
>
> We encountered a problem when using 
> {{AtlasEntityStore.getByUniqueAttributes}}. One Hive database has 3000+ 
> tables, getting metadata of the database by 
> {{AtlasEntityStore.getByUniqueAttributes}} costs more than 1 seconds.
> After investigation, we realized that 
> {{AtlasEntityStore.getByUniqueAttributes}} wil return all attributes and 
> relationshipAttributes of the entity.  In actuality we need'n to get 
> relationshiAttributes like tables.
> We propose to add a parameter {{ignoreRelationship}} to 
> {{AtlasEntityStore.getByUniqueAttributes}}. When set to true, the functions 
> only return attributes of the entity and ignore relationshipAttributes. We 
> also add the parameter to {{AtlasClientV2}} and {{EntityREST}} accordingly.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3099) Update graph storage and index backend property to be configurable

2019-03-25 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801216#comment-16801216
 ] 

Sarath Subramanian commented on ATLAS-3099:
---

Code review: [https://reviews.apache.org/r/70301/]

> Update graph storage and index backend property to be configurable
> --
>
> Key: ATLAS-3099
> URL: https://issues.apache.org/jira/browse/ATLAS-3099
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> In ApplicationProperties the values of the following properties are hardcoded 
> to:
>  * atlas.graph.storage.backend=hbase2
>  * atlas.graph.index.search.backend=solr
> This should be made configurable and user provided values for these 
> properties need to be honored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3099) Update graph storage and index backend property to be configurable

2019-03-25 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3099:
--
Description: 
In ApplicationProperties the values of the following properties are hardcoded 
to:
 * atlas.graph.storage.backend=hbase2
 * atlas.graph.index.search.backend=solr

This should be made configurable and user provided values for these properties 
need to be honored.

  was:
In ApplicationProperties the values of the following properties are hardcoded 
to:
 * atlas.graph.storage.backend=hbase2
 * atlas.graph.index.search.backend=solr

This should be made configurable and user supplied values for these properties 
needs to be honored.


> Update graph storage and index backend property to be configurable
> --
>
> Key: ATLAS-3099
> URL: https://issues.apache.org/jira/browse/ATLAS-3099
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> In ApplicationProperties the values of the following properties are hardcoded 
> to:
>  * atlas.graph.storage.backend=hbase2
>  * atlas.graph.index.search.backend=solr
> This should be made configurable and user provided values for these 
> properties need to be honored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3099) Update graph storage and index backend property to be configurable

2019-03-25 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3099:
-

 Summary: Update graph storage and index backend property to be 
configurable
 Key: ATLAS-3099
 URL: https://issues.apache.org/jira/browse/ATLAS-3099
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0, trunk


In ApplicationProperties the values of the following properties are hardcoded 
to:
 * atlas.graph.storage.backend=hbase2
 * atlas.graph.index.search.backend=solr

This should be made configurable and user supplied values for these properties 
needs to be honored.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3095) Update QuickStartV2 to use relationships

2019-03-25 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3095:
--
Summary: Update QuickStartV2 to use relationships  (was: update 
QuickStartV2 to use relationships)

> Update QuickStartV2 to use relationships
> 
>
> Key: ATLAS-3095
> URL: https://issues.apache.org/jira/browse/ATLAS-3095
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0, trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> QuickstartV2 doesn't uses relationships to create entities. This Jira adds 
> new relationshipDefs and creates sample entities using relationship.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3095) Update QuickStartV2 to use relationships

2019-03-25 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16800433#comment-16800433
 ] 

Sarath Subramanian commented on ATLAS-3095:
---

Code review: [https://reviews.apache.org/r/70292/]

> Update QuickStartV2 to use relationships
> 
>
> Key: ATLAS-3095
> URL: https://issues.apache.org/jira/browse/ATLAS-3095
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0, trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> QuickstartV2 doesn't uses relationships to create entities. This Jira adds 
> new relationshipDefs and creates sample entities using relationship.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3095) update QuickStartV2 to use relationships

2019-03-24 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3095:
-

 Summary: update QuickStartV2 to use relationships
 Key: ATLAS-3095
 URL: https://issues.apache.org/jira/browse/ATLAS-3095
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0, trunk
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0, trunk


QuickstartV2 doesn't uses relationships to create entities. This Jira adds new 
relationshipDefs and creates sample entities using relationship.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (ATLAS-2933) Empty array attributes are returned as null instead of an empty list

2019-04-03 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian resolved ATLAS-2933.
---
Resolution: Fixed

> Empty array attributes are returned as null instead of an empty list
> 
>
> Key: ATLAS-2933
> URL: https://issues.apache.org/jira/browse/ATLAS-2933
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Cade Parker
>Assignee: Le Ma
>Priority: Major
> Attachments: ATLAS-2933.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
>  
> Given the following entity def:
> {code:java}
> {
>  "entityDefs": [
>{
>  "name": "ExampleType",
>  "superTypes": [],
>  "typeVersion": "0.1",
>  "attributeDefs": [
>{
>  "name": "things",
>  "typeName": "array",
>  "cardinality": "SINGLE",
>  "isIndexable": false,
>  "isOptional": false,
>  "isUnique": false
>}
>  ]
>}
>  ]
> }
> {code}
> I can POST a new entity with an empty array for the things attribute 
> successfully:
> {code:java}
> POST /api/atlas/v2/entity
> {
>  "entity":{
>  "typeName":"ExampleType",
>  "attributes":{
>"things": []
>  },
>  "guid":"-thing",
>  "version":1
>  }
> }
> {code}
> But when i GET the entity by guid, the attribute is null instead of the empty 
> list:
> {code:java}
> GET /api/atlas/v2/entity/guid/d5e33187-3293-4206-bcb8-807dc87c1440
> {
>  "referredEntities": {},
>  "entity": {
>  "typeName": "ExampleType",
>  "attributes": {
>"things": null
>  },
>  "guid": "d5e33187-3293-4206-bcb8-807dc87c1440",
>  "status": "ACTIVE",
>  "createdBy": "admin",
>  "updatedBy": "admin",
>  "createTime": 1540335741854,
>  "updateTime": 1540335741854,
>  "version": 1
>  }
> }
> {code}
> Atlas should preserve the original value of the create request as an empty 
> list. Returning null here causes further confusion, because the attribute is 
> defined with `isOptional: false` so any consumer of the entity would 
> reasonably expect a non-null value.
> I tested this behavior against Atlas versions 1.0.0 and 1.1.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3017) Add Atlas server statistics rest endpoint

2019-04-05 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811139#comment-16811139
 ] 

Sarath Subramanian commented on ATLAS-3017:
---

+1 for the patch ATLAS-3017-Atlas-server-statistics-1.patch

> Add Atlas server statistics rest endpoint 
> --
>
> Key: ATLAS-3017
> URL: https://issues.apache.org/jira/browse/ATLAS-3017
> Project: Atlas
>  Issue Type: Bug
>Reporter: Nixon Rodrigues
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-3017-Atlas-server-statistics-1.patch, 
> ATLAS-3017-Atlas-server-statistics.patch, ATLAS-3017.patch
>
>
> Provide rest endpoint that will return server statistics e.g. time since 
> server is up, time at last kafka message processed, kafka offset related 
> info, hbase state.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3104) Fix stale transaction alerts in atlas logs

2019-03-28 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3104:
-

 Summary: Fix stale transaction alerts in atlas logs
 Key: ATLAS-3104
 URL: https://issues.apache.org/jira/browse/ATLAS-3104
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0, trunk
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


stale transaction alerts have been seen in atlas logs after the java patch 
handler.

Need to close transaction after graph access.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3122) Publish release artifacts

2019-04-07 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3122:
-

 Summary: Publish release artifacts
 Key: ATLAS-3122
 URL: https://issues.apache.org/jira/browse/ATLAS-3122
 Project: Atlas
  Issue Type: Sub-task
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3121) Change version from 2.0.0-SNAPSHOT to 2.0.0

2019-04-07 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3121:
-

 Summary: Change version from 2.0.0-SNAPSHOT to 2.0.0
 Key: ATLAS-3121
 URL: https://issues.apache.org/jira/browse/ATLAS-3121
 Project: Atlas
  Issue Type: Sub-task
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian


Update pom.xml files to replace version 2.0.0-SNAPSHOT to 2.0.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3120) Release Atlas 2.0.0

2019-04-07 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3120:
-

 Summary: Release Atlas 2.0.0
 Key: ATLAS-3120
 URL: https://issues.apache.org/jira/browse/ATLAS-3120
 Project: Atlas
  Issue Type: Task
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3123) Update Atlas website for 2.0.0 release

2019-04-07 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3123:
-

 Summary: Update Atlas website for 2.0.0 release
 Key: ATLAS-3123
 URL: https://issues.apache.org/jira/browse/ATLAS-3123
 Project: Atlas
  Issue Type: Sub-task
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3125) Update version in branch-2.0 to 2.1.0-SNAPSHOT

2019-04-07 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3125:
--
Issue Type: Task  (was: Sub-task)
Parent: (was: ATLAS-2688)

> Update version in branch-2.0 to 2.1.0-SNAPSHOT
> --
>
> Key: ATLAS-3125
> URL: https://issues.apache.org/jira/browse/ATLAS-3125
> Project: Atlas
>  Issue Type: Task
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> After Apache Atlas 2.0.0 has been released, update branch-2.0 to use version 
> 2.1.0-SNAPSHOT



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3124) Update version in master branch to 3.0.0-SNAPSHOT

2019-04-07 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3124:
-

 Summary: Update version in master branch to 3.0.0-SNAPSHOT
 Key: ATLAS-3124
 URL: https://issues.apache.org/jira/browse/ATLAS-3124
 Project: Atlas
  Issue Type: Sub-task
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


After Apache Atlas 2.0.0 has been released, update master branch to use version 
3.0.0-SNAPSHOT



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3125) Update version in branch-2.0 to 2.1.0-SNAPSHOT

2019-04-07 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3125:
-

 Summary: Update version in branch-2.0 to 2.1.0-SNAPSHOT
 Key: ATLAS-3125
 URL: https://issues.apache.org/jira/browse/ATLAS-3125
 Project: Atlas
  Issue Type: Sub-task
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


After Apache Atlas 2.0.0 has been released, update branch-2.0 to use version 
2.1.0-SNAPSHOT



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3125) Update version in branch-2.0 to 2.1.0-SNAPSHOT

2019-04-07 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3125:
--
Issue Type: Sub-task  (was: Task)
Parent: ATLAS-3120

> Update version in branch-2.0 to 2.1.0-SNAPSHOT
> --
>
> Key: ATLAS-3125
> URL: https://issues.apache.org/jira/browse/ATLAS-3125
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> After Apache Atlas 2.0.0 has been released, update branch-2.0 to use version 
> 2.1.0-SNAPSHOT



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3120) Release Atlas 2.0.0

2019-04-07 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3120:
--
Description: parent task to capture release tasks for Apache Atlas 2.0

> Release Atlas 2.0.0
> ---
>
> Key: ATLAS-3120
> URL: https://issues.apache.org/jira/browse/ATLAS-3120
> Project: Atlas
>  Issue Type: Task
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> parent task to capture release tasks for Apache Atlas 2.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3112) Allow Indexing of array attributes (LIST or SET) in indexing store

2019-04-07 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811978#comment-16811978
 ] 

Sarath Subramanian commented on ATLAS-3112:
---

Reverting this commit. This is limitation in jansugraph. 

Limited Mixed Index Support
{quote} 
Mixed indexes only support a subset of the data types that JanusGraph supports. 
See [Mixed Index Data 
Types|https://docs.janusgraph.org/0.3.1/search-predicates.html#mixeddatatypes] 
for a current listing. Also, mixed indexes do not currently support property 
keys with SET or LIST cardinality.
{quote}

> Allow Indexing of array attributes (LIST or SET) in indexing store
> --
>
> Key: ATLAS-3112
> URL: https://issues.apache.org/jira/browse/ATLAS-3112
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> Array attributes with cardinality LIST or SET is not indexed in the indexing 
> store. To allow index query to search on array attributes, array attributes 
> needs to be indexed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3059) UI : Lineage data structure change to support hide deleted and hide process entities to gather

2019-02-27 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779760#comment-16779760
 ] 

Sarath Subramanian commented on ATLAS-3059:
---

+1 for the patch. Looks good Keval. Thanks.

> UI : Lineage data structure change to support hide deleted and hide process 
> entities to gather
> --
>
> Key: ATLAS-3059
> URL: https://issues.apache.org/jira/browse/ATLAS-3059
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 0.8.4, 1.2.0, 2.0.0
>
> Attachments: ATLAS-3059.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3031) UI : Allow user to export the lineage in PNG or JPEG format

2019-03-20 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797469#comment-16797469
 ] 

Sarath Subramanian commented on ATLAS-3031:
---

+1 for the patch. Thanks [~binitgutka].

> UI : Allow user to export the lineage in PNG or JPEG format
> ---
>
> Key: ATLAS-3031
> URL: https://issues.apache.org/jira/browse/ATLAS-3031
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Binit Gutka
>Priority: Major
> Attachments: ATLAS-3031.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3077) Handle java patches in patch framework

2019-03-18 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16795469#comment-16795469
 ] 

Sarath Subramanian commented on ATLAS-3077:
---

Code review: [https://reviews.apache.org/r/70236/]

> Handle java patches in patch framework 
> ---
>
> Key: ATLAS-3077
> URL: https://issues.apache.org/jira/browse/ATLAS-3077
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>
> We have Atlas typedef patches available to update out of box type 
> definitions. This JIRA will introduce a new java patch framework to handle 
> changes to the graph database not specific to type changes.
> For e.g. In case of migration, to add/update property keys in vertices - java 
> patch framework can be used, which will be run at the atlas startup time 
> during bootstrap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3073) Investigate and fix IT failures in EntityJerseyResourceIT

2019-03-12 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790837#comment-16790837
 ] 

Sarath Subramanian commented on ATLAS-3073:
---

Code review: https://reviews.apache.org/r/70197/

> Investigate and fix IT failures in EntityJerseyResourceIT
> -
>
> Key: ATLAS-3073
> URL: https://issues.apache.org/jira/browse/ATLAS-3073
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
> Attachments: 
> 0001-ATLAS-3073-Investigate-and-fix-IT-failures-in-Entity.patch
>
>
> Investigate and fix IT failures in EntityJerseyResourceIT:
> Failed tests:
>  EntityJerseyResourceIT.testCompleteUpdate:1001 Maps do not have the same 
> size:11 != 8
>  EntityJerseyResourceIT.testPartialUpdateByGuid:889 Maps do not have the same 
> size:8 != 11
>  EntityJerseyResourceIT.testPartialUpdateByUniqueAttributes:944 expected 
> [true] but found [false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3073) Investigate and fix IT failures in EntityJerseyResourceIT

2019-03-12 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3073:
-

 Summary: Investigate and fix IT failures in EntityJerseyResourceIT
 Key: ATLAS-3073
 URL: https://issues.apache.org/jira/browse/ATLAS-3073
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0, trunk


Investigate and fix IT failures in EntityJerseyResourceIT:

Failed tests:
 EntityJerseyResourceIT.testCompleteUpdate:1001 Maps do not have the same 
size:11 != 8
 EntityJerseyResourceIT.testPartialUpdateByGuid:889 Maps do not have the same 
size:8 != 11
 EntityJerseyResourceIT.testPartialUpdateByUniqueAttributes:944 expected [true] 
but found [false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3073) Investigate and fix IT failures in EntityJerseyResourceIT

2019-03-12 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3073:
--
Attachment: 0001-ATLAS-3073-Investigate-and-fix-IT-failures-in-Entity.patch

> Investigate and fix IT failures in EntityJerseyResourceIT
> -
>
> Key: ATLAS-3073
> URL: https://issues.apache.org/jira/browse/ATLAS-3073
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
> Attachments: 
> 0001-ATLAS-3073-Investigate-and-fix-IT-failures-in-Entity.patch
>
>
> Investigate and fix IT failures in EntityJerseyResourceIT:
> Failed tests:
>  EntityJerseyResourceIT.testCompleteUpdate:1001 Maps do not have the same 
> size:11 != 8
>  EntityJerseyResourceIT.testPartialUpdateByGuid:889 Maps do not have the same 
> size:8 != 11
>  EntityJerseyResourceIT.testPartialUpdateByUniqueAttributes:944 expected 
> [true] but found [false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3077) Handle java patches in patch framework

2019-03-15 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3077:
-

 Summary: Handle java patches in patch framework 
 Key: ATLAS-3077
 URL: https://issues.apache.org/jira/browse/ATLAS-3077
 Project: Atlas
  Issue Type: Bug
Reporter: Sarath Subramanian


We have Atlas typedef patches available to update out of box type definitions. 
This JIRA will introduce a new java patch framework to handle changes to the 
graph database not specific to type changes.

For e.g. In case of migration, to add/update property keys in vertices - java 
patch framework can be used, which will be run at the atlas startup time during 
bootstrap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (ATLAS-3077) Handle java patches in patch framework

2019-03-15 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian reassigned ATLAS-3077:
-

Assignee: Sarath Subramanian

> Handle java patches in patch framework 
> ---
>
> Key: ATLAS-3077
> URL: https://issues.apache.org/jira/browse/ATLAS-3077
> Project: Atlas
>  Issue Type: Bug
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>
> We have Atlas typedef patches available to update out of box type 
> definitions. This JIRA will introduce a new java patch framework to handle 
> changes to the graph database not specific to type changes.
> For e.g. In case of migration, to add/update property keys in vertices - java 
> patch framework can be used, which will be run at the atlas startup time 
> during bootstrap.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3070) Create patch framework to persist typedef patches applied to atlas

2019-03-11 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3070:
--
Summary: Create patch framework to persist typedef patches applied to atlas 
 (was: Create patch framework to keep track of typedef patches applied to atlas)

> Create patch framework to persist typedef patches applied to atlas
> --
>
> Key: ATLAS-3070
> URL: https://issues.apache.org/jira/browse/ATLAS-3070
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.1.0, trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> In Atlas, we don't have a way to see all the type patches applied. The patch 
> framework will create a new vertex in the graph database and update its 
> application state. This will be useful when applying new patches and to make 
> sure not to apply applied patches again. A new REST API is also made 
> available to display all the list of patches and its application status.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3070) Create patch framework to keep track of typedef patches applied to atlas

2019-03-11 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3070:
-

 Summary: Create patch framework to keep track of typedef patches 
applied to atlas
 Key: ATLAS-3070
 URL: https://issues.apache.org/jira/browse/ATLAS-3070
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 1.1.0, trunk
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.0.0


In Atlas, we don't have a way to see all the type patches applied. The patch 
framework will create a new vertex in the graph database and update its 
application state. This will be useful when applying new patches and to make 
sure not to apply applied patches again. A new REST API is also made available 
to display all the list of patches and its application status.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3070) Create patch framework to persist typedef patches applied to atlas

2019-03-11 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789926#comment-16789926
 ] 

Sarath Subramanian commented on ATLAS-3070:
---

Code review: https://reviews.apache.org/r/70188/

> Create patch framework to persist typedef patches applied to atlas
> --
>
> Key: ATLAS-3070
> URL: https://issues.apache.org/jira/browse/ATLAS-3070
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.1.0, trunk
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> In Atlas, we don't have a way to see all the type patches applied. The patch 
> framework will create a new vertex in the graph database and update its 
> application state. This will be useful when applying new patches and to make 
> sure not to apply applied patches again. A new REST API is also made 
> available to display all the list of patches and its application status.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (ATLAS-2458) RelationshipApi Delete doesnt remove relationshipAttributes from entity

2019-04-09 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian resolved ATLAS-2458.
---
Resolution: Fixed

> RelationshipApi Delete doesnt remove relationshipAttributes from entity
> ---
>
> Key: ATLAS-2458
> URL: https://issues.apache.org/jira/browse/ATLAS-2458
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Brandon Kirchner
>Assignee: Kapildeo Nayak
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2458.patch
>
>
> When I create a Process and DataSet entity, and create a 
> process_dataset_outputs relationship between the two, when I delete the 
> relationship, the relationshipAttributes are not removed from the Process or 
> DataSet entities. full reproduction steps and output are below --
>  
> step 1 -- create dataset and process (through UI)
> step 2 -- create relationship
> http post http://localhost:21000/api/atlas/v2/relationship
> PAYLOAD
> {
>  "end1" : {
>  "guid" : "d456dc93-8832-4a36-8ae0-071946c653fc"
>  },
>  "end2" : {
>  "guid" : "247bc2d1-de27-4c50-a35b-b62350dc7979"
>  },
>  "typeName":"process_dataset_outputs"
> }
> RESPONSE
> {
>  "createTime": 1519335073389,
>  "createdBy": "admin",
>  "end1": {
>  "guid": "d456dc93-8832-4a36-8ae0-071946c653fc",
>  "typeName": "Process"
>  },
>  "end2": {
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "typeName": "DataSet"
>  },
>  "guid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "label": "__Process.outputs",
>  "propagateTags": "ONE_TO_TWO",
>  "status": "ACTIVE",
>  "typeName": "process_dataset_outputs",
>  "updateTime": 1519335073389,
>  "updatedBy": "admin",
>  "version": 0
> }
> step 3 -- verify relationship on entities
> http get 
> http://localhost:21000/api/atlas/v2/entity/guid/247bc2d1-de27-4c50-a35b-b62350dc7979
> HTTP/1.1 200 OK
> {
>  "entity": {
>  "attributes": {
>  "description": null,
>  "name": "dataset",
>  "owner": null,
>  "qualifiedName": "dataset"
>  },
>  "classifications": [],
>  "createTime": 1519334042139,
>  "createdBy": "admin",
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "relationshipAttributes": {
>  "sinkFromProcesses": [
>  {
>  "displayText": "process",
>  "guid": "d456dc93-8832-4a36-8ae0-071946c653fc",
>  "relationshipAttributes": {
>  "typeName": "process_dataset_outputs"
>  },
>  "relationshipGuid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "typeName": "Process"
>  }
>  ],
>  "sourceToProcesses": []
>  },
>  "status": "ACTIVE",
>  "typeName": "DataSet",
>  "updateTime": 1519334042139,
>  "updatedBy": "admin",
>  "version": 0
>  },
>  "referredEntities": {}
> }
> http get 
> http://localhost:21000/api/atlas/v2/entity/guid/247bc2d1-de27-4c50-a35b-b62350dc7979
> HTTP/1.1 200 OK
> {
>  "entity": {
>  "attributes": {
>  "description": null,
>  "name": "dataset",
>  "owner": null,
>  "qualifiedName": "dataset"
>  },
>  "classifications": [],
>  "createTime": 1519334042139,
>  "createdBy": "admin",
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "relationshipAttributes": {
>  "sinkFromProcesses": [
>  {
>  "displayText": "process",
>  "guid": "d456dc93-8832-4a36-8ae0-071946c653fc",
>  "relationshipAttributes": {
>  "typeName": "process_dataset_outputs"
>  },
>  "relationshipGuid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "typeName": "Process"
>  }
>  ],
>  "sourceToProcesses": []
>  },
>  "status": "ACTIVE",
>  "typeName": "DataSet",
>  "updateTime": 1519334042139,
>  "updatedBy": "admin",
>  "version": 0
>  },
>  "referredEntities": {}
> }
> step 4 -- delete relationship
> http delete 
> http://localhost:21000/api/atlas/v2/relationship/guid/bb9dfa43-c193-4cbb-9472-6b9421f43701
> HTTP/1.1 204 No Content
> step 5 -- verify deletion
> http get 
> http://localhost:21000/api/atlas/v2/relationship/guid/bb9dfa43-c193-4cbb-9472-6b9421f43701
> HTTP/1.1 200 OK
> {
>  "createTime": 1519334256264,
>  "createdBy": "admin",
>  "end1": {
>  "guid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "typeName": "Process"
>  },
>  "end2": {
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "typeName": "DataSet"
>  },
>  "guid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "label": "__Process.outputs",
>  "propagateTags": "ONE_TO_TWO",
>  "status": "DELETED",
>  "typeName": "process_dataset_outputs",
>  "updateTime": 1519334383683,
>  "updatedBy": "admin",
>  "version": 0
> }
> step 6 -- verify relationship removed from entities (FAILS)
> http get 
> http://localhost:21000/api/atlas/v2/entity/guid/247bc2d1-de27-4c50-a35b-b62350dc7979
> HTTP/1.1 200 OK
> {
>  "entity": {
>  "attributes": {
>  "description": null,
>  "name": "dataset",
>  "owner": null,
>  "qualifiedName": "dataset"
>  },
>  "classifications": [],
>  "createTime": 1519334042139,
>  "createdBy": "admin",
>  "guid": 

[jira] [Updated] (ATLAS-2458) RelationshipApi Delete doesnt remove relationshipAttributes from entity

2019-04-09 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-2458:
--
Fix Version/s: 2.0.0

> RelationshipApi Delete doesnt remove relationshipAttributes from entity
> ---
>
> Key: ATLAS-2458
> URL: https://issues.apache.org/jira/browse/ATLAS-2458
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Brandon Kirchner
>Assignee: Kapildeo Nayak
>Priority: Major
> Fix For: 1.0.0, 2.0.0
>
> Attachments: ATLAS-2458.patch
>
>
> When I create a Process and DataSet entity, and create a 
> process_dataset_outputs relationship between the two, when I delete the 
> relationship, the relationshipAttributes are not removed from the Process or 
> DataSet entities. full reproduction steps and output are below --
>  
> step 1 -- create dataset and process (through UI)
> step 2 -- create relationship
> http post http://localhost:21000/api/atlas/v2/relationship
> PAYLOAD
> {
>  "end1" : {
>  "guid" : "d456dc93-8832-4a36-8ae0-071946c653fc"
>  },
>  "end2" : {
>  "guid" : "247bc2d1-de27-4c50-a35b-b62350dc7979"
>  },
>  "typeName":"process_dataset_outputs"
> }
> RESPONSE
> {
>  "createTime": 1519335073389,
>  "createdBy": "admin",
>  "end1": {
>  "guid": "d456dc93-8832-4a36-8ae0-071946c653fc",
>  "typeName": "Process"
>  },
>  "end2": {
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "typeName": "DataSet"
>  },
>  "guid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "label": "__Process.outputs",
>  "propagateTags": "ONE_TO_TWO",
>  "status": "ACTIVE",
>  "typeName": "process_dataset_outputs",
>  "updateTime": 1519335073389,
>  "updatedBy": "admin",
>  "version": 0
> }
> step 3 -- verify relationship on entities
> http get 
> http://localhost:21000/api/atlas/v2/entity/guid/247bc2d1-de27-4c50-a35b-b62350dc7979
> HTTP/1.1 200 OK
> {
>  "entity": {
>  "attributes": {
>  "description": null,
>  "name": "dataset",
>  "owner": null,
>  "qualifiedName": "dataset"
>  },
>  "classifications": [],
>  "createTime": 1519334042139,
>  "createdBy": "admin",
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "relationshipAttributes": {
>  "sinkFromProcesses": [
>  {
>  "displayText": "process",
>  "guid": "d456dc93-8832-4a36-8ae0-071946c653fc",
>  "relationshipAttributes": {
>  "typeName": "process_dataset_outputs"
>  },
>  "relationshipGuid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "typeName": "Process"
>  }
>  ],
>  "sourceToProcesses": []
>  },
>  "status": "ACTIVE",
>  "typeName": "DataSet",
>  "updateTime": 1519334042139,
>  "updatedBy": "admin",
>  "version": 0
>  },
>  "referredEntities": {}
> }
> http get 
> http://localhost:21000/api/atlas/v2/entity/guid/247bc2d1-de27-4c50-a35b-b62350dc7979
> HTTP/1.1 200 OK
> {
>  "entity": {
>  "attributes": {
>  "description": null,
>  "name": "dataset",
>  "owner": null,
>  "qualifiedName": "dataset"
>  },
>  "classifications": [],
>  "createTime": 1519334042139,
>  "createdBy": "admin",
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "relationshipAttributes": {
>  "sinkFromProcesses": [
>  {
>  "displayText": "process",
>  "guid": "d456dc93-8832-4a36-8ae0-071946c653fc",
>  "relationshipAttributes": {
>  "typeName": "process_dataset_outputs"
>  },
>  "relationshipGuid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "typeName": "Process"
>  }
>  ],
>  "sourceToProcesses": []
>  },
>  "status": "ACTIVE",
>  "typeName": "DataSet",
>  "updateTime": 1519334042139,
>  "updatedBy": "admin",
>  "version": 0
>  },
>  "referredEntities": {}
> }
> step 4 -- delete relationship
> http delete 
> http://localhost:21000/api/atlas/v2/relationship/guid/bb9dfa43-c193-4cbb-9472-6b9421f43701
> HTTP/1.1 204 No Content
> step 5 -- verify deletion
> http get 
> http://localhost:21000/api/atlas/v2/relationship/guid/bb9dfa43-c193-4cbb-9472-6b9421f43701
> HTTP/1.1 200 OK
> {
>  "createTime": 1519334256264,
>  "createdBy": "admin",
>  "end1": {
>  "guid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "typeName": "Process"
>  },
>  "end2": {
>  "guid": "247bc2d1-de27-4c50-a35b-b62350dc7979",
>  "typeName": "DataSet"
>  },
>  "guid": "bb9dfa43-c193-4cbb-9472-6b9421f43701",
>  "label": "__Process.outputs",
>  "propagateTags": "ONE_TO_TWO",
>  "status": "DELETED",
>  "typeName": "process_dataset_outputs",
>  "updateTime": 1519334383683,
>  "updatedBy": "admin",
>  "version": 0
> }
> step 6 -- verify relationship removed from entities (FAILS)
> http get 
> http://localhost:21000/api/atlas/v2/entity/guid/247bc2d1-de27-4c50-a35b-b62350dc7979
> HTTP/1.1 200 OK
> {
>  "entity": {
>  "attributes": {
>  "description": null,
>  "name": "dataset",
>  "owner": null,
>  "qualifiedName": "dataset"
>  },
>  "classifications": [],
>  "createTime": 1519334042139,
>  "createdBy": "admin",
>  "guid": 

[jira] [Commented] (ATLAS-3116) Fix LDAP login, once a sign-on is made through knox SSO.

2019-04-09 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813024#comment-16813024
 ] 

Sarath Subramanian commented on ATLAS-3116:
---

+1 for patch - ATLAS-3116.patch

 

> Fix LDAP login, once a sign-on is made through knox SSO.
> 
>
> Key: ATLAS-3116
> URL: https://issues.apache.org/jira/browse/ATLAS-3116
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.8.3, 1.1.0
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 0.8.4, 1.2.0, 2.0.0
>
> Attachments: ATLAS-3116.patch
>
>
> *Background of the issue.*
>  
> Dependency of AtlasAuthenticationProvider is Injected in two place from 
> constructor with @Inject 1) 
> [AtlasKnoxSSOFilter|https://github.com/apache/atlas/blob/master/webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java#L102]
>  and 
> [AtlasSecurityConfig|https://github.com/apache/atlas/blob/master/webapp/src/main/java/org/apache/atlas/web/security/AtlasSecurityConfig.java#L72].
> After observing the hashCode of objects of the AtlasAuthenticationProvider 
> appeared to be same in respectively classes.
>  
> If no scope is specified single instance per container is created and thats 
> caused the bug described below.
>  
> The *ssoEnabled* is a property in AtlasAuthenticationProvider which set from 
> AtlasKnoxSSOFilter flow. Since the object reference were same, once the 
> ssoEnabled is set true,  it apply for LdapAuthentication flow also and the 
> [ldap authentication flow is goes in sso 
> block|https://github.com/apache/atlas/blob/master/webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationProvider.java#L92]
>  
> To fix the issue added scope prototype to AtlasAuthenticationProvider , so 
> that different objects are create in AtlasKnoxSSOFilter and 
> AtlasSecurityConfig respectively.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3136) assigning a term to an invalid entity guid throws error 500, instead of 400

2019-04-14 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16817529#comment-16817529
 ] 

Sarath Subramanian commented on ATLAS-3136:
---

+1 for the patch. Thanks [~madhan.neethiraj].

> assigning a term to an invalid entity guid throws error 500, instead of 400
> ---
>
> Key: ATLAS-3136
> URL: https://issues.apache.org/jira/browse/ATLAS-3136
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sharmadha Sainath
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-3136.patch
>
>
> Calling REST API : /api/atlas/v2/glossary/terms//assignedEntities, 
> with following body throws error code 500 (internal error) instead of 400 
> (bad request):
> {noformat}
> [{"guid": "invalid_guid"}]
> {noformat}
> Following error is logged by Atlas server:
> {noformat}
> graph rollback due to exception  (GraphTransactionInterceptor:164)
> java.lang.NullPointerException
>   at 
> org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.getTypeName(AtlasGraphUtilsV2.java:140)
>   at 
> org.apache.atlas.repository.store.graph.v2.AtlasRelationshipStoreV2.getRelationshipEdgeLabel(AtlasRelationshipStoreV2.java:805)
>   at 
> org.apache.atlas.repository.store.graph.v2.AtlasRelationshipStoreV2.getRelationship(AtlasRelationshipStoreV2.java:298)
>   at 
> org.apache.atlas.repository.store.graph.v2.AtlasRelationshipStoreV2.getOrCreate(AtlasRelationshipStoreV2.java:314)
>   at 
> org.apache.atlas.repository.store.graph.v2.AtlasRelationshipStoreV2$$FastClassBySpringCGLIB$$a8165974.invoke()
>   at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>   at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:667)
>   at 
> org.apache.atlas.repository.store.graph.v2.AtlasRelationshipStoreV2$$EnhancerBySpringCGLIB$$b5f3df70.getOrCreate()
>   at 
> org.apache.atlas.glossary.GlossaryUtils.createRelationship(GlossaryUtils.java:91)
>   at 
> org.apache.atlas.glossary.GlossaryTermUtils.processTermAssignments(GlossaryTermUtils.java:90)
>   at 
> org.apache.atlas.glossary.GlossaryService.assignTermToEntities(GlossaryService.java:481)
>   at 
> org.apache.atlas.glossary.GlossaryService$$FastClassBySpringCGLIB$$e1f893e0.invoke()
>   at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
>   at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>   at 
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:80)
>   at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>   at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
>   at 
> org.apache.atlas.glossary.GlossaryService$$EnhancerBySpringCGLIB$$236c7234.assignTermToEntities()
>   at 
> org.apache.atlas.web.rest.GlossaryREST.assignTermToEntities(GlossaryREST.java:842)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>   at 
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$VoidOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:167)
>   at 
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
>   at 
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
>   at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>   at 
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
>   at 
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>   at 
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
>   at 
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
>   at 
> 

[jira] [Commented] (ATLAS-3148) Implement Hive Metastore hook for Atlas

2019-04-17 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820578#comment-16820578
 ] 

Sarath Subramanian commented on ATLAS-3148:
---

Code review: [https://reviews.apache.org/r/70485/]

> Implement Hive Metastore hook for Atlas
> ---
>
> Key: ATLAS-3148
> URL: https://issues.apache.org/jira/browse/ATLAS-3148
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which 
> registers with HiveServer2 process to listen for create/update/delete 
> operations and updates the metadata in Atlas.
> If hive metastore is accessed using other clients like Impala shell, Hue or 
> other JDBC/ODBC client apps there is no way to capture the events in Atlas.
> This Jira will create a new atlas hook for Hive Metastore - stores the 
> metadata for Hive tables and partitions in a relational database, and 
> provides clients (including Hive) access to this information using the 
> metastore service API. 
> This hook is registered as post-listener (*hive.metastore.event.listeners*) 
> and DDL operations are captured and send to atlas kafka topic for processing 
> by Atlas server.
> The following DDL operations are captured:
>  * *CreateDatabaseEvent*
>  * *DropDatabaseEvent*
>  * *AlterDatabaseEvent*
>  * *CreateTableEvent*
>  * *DropTableEvent*
>  * *AlterTableEvent*
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3148) Implement Hive Metastore hook for Atlas

2019-04-17 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3148:
--
Description: 
Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which registers 
with HiveServer2 process to listen for create/update/delete operations and 
updates the metadata in Atlas.

If hive metastore is accessed using other clients like Impala shell, Hue or 
other JDBC/ODBC client apps there is no way to capture the events in Atlas.

This Jira will create a new atlas hook for Hive Metastore - stores the metadata 
for Hive tables and partitions in a relational database, and provides clients 
(including Hive) access to this information using the metastore service API. 

This hook is registered as post-listener (hive.metastore.event.listeners) and 
DDL operations are captured and send to atlas kafka topic for processing by 
Atlas server.

The following DDL operations are captured:
 * CreateDatabaseEvent
 * DropDatabaseEvent
 * AlterDatabaseEvent
 * CreateTableEvent
 * DropTableEvent
 * AlterTableEvent

 

  was:
Atlas has [hive hook|[http://atlas.apache.org/Hook-Hive.html]] which registers 
with HiveServer2 process to listen for create/update/delete operations and 
updates the metadata in Atlas.

If hive metastore is accessed using other clients like Impala shell, Hue or 
other JDBC/ODBC client apps there is no way to capture the events in Atlas.

This Jira will create a new atlas hook for Hive Metastore - stores the metadata 
for Hive tables and partitions in a relational database, and provides clients 
(including Hive) access to this information using the metastore service API. 

This hook is registered as post-listener (hive.metastore.event.listeners) and 
DDL operations are captured and send to atlas kafka topic for processing by 
Atlas server.

The following DDL operations are captured:
 * CreateDatabaseEvent
 * DropDatabaseEvent
 * AlterDatabaseEvent
 * CreateTableEvent
 * DropTableEvent
 * AlterTableEvent

 


> Implement Hive Metastore hook for Atlas
> ---
>
> Key: ATLAS-3148
> URL: https://issues.apache.org/jira/browse/ATLAS-3148
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which 
> registers with HiveServer2 process to listen for create/update/delete 
> operations and updates the metadata in Atlas.
> If hive metastore is accessed using other clients like Impala shell, Hue or 
> other JDBC/ODBC client apps there is no way to capture the events in Atlas.
> This Jira will create a new atlas hook for Hive Metastore - stores the 
> metadata for Hive tables and partitions in a relational database, and 
> provides clients (including Hive) access to this information using the 
> metastore service API. 
> This hook is registered as post-listener (hive.metastore.event.listeners) and 
> DDL operations are captured and send to atlas kafka topic for processing by 
> Atlas server.
> The following DDL operations are captured:
>  * CreateDatabaseEvent
>  * DropDatabaseEvent
>  * AlterDatabaseEvent
>  * CreateTableEvent
>  * DropTableEvent
>  * AlterTableEvent
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3148) Implement Hive Metastore hook for Atlas

2019-04-17 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3148:
--
Description: 
Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which registers 
with HiveServer2 process to listen for create/update/delete operations and 
updates the metadata in Atlas.

If hive metastore is accessed using other clients like Impala shell, Hue or 
other JDBC/ODBC client apps there is no way to capture the events in Atlas.

This Jira will create a new atlas hook for Hive Metastore - stores the metadata 
for Hive tables and partitions in a relational database, and provides clients 
(including Hive) access to this information using the metastore service API. 

This hook is registered as post-listener (*hive.metastore.event.listeners*) and 
DDL operations are captured and send to atlas kafka topic for processing by 
Atlas server.

The following DDL operations are captured:
 * *CreateDatabaseEvent*
 * *DropDatabaseEvent*
 * *AlterDatabaseEvent*
 * *CreateTableEvent*
 * *DropTableEvent*
 * *AlterTableEvent*

 

  was:
Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which registers 
with HiveServer2 process to listen for create/update/delete operations and 
updates the metadata in Atlas.

If hive metastore is accessed using other clients like Impala shell, Hue or 
other JDBC/ODBC client apps there is no way to capture the events in Atlas.

This Jira will create a new atlas hook for Hive Metastore - stores the metadata 
for Hive tables and partitions in a relational database, and provides clients 
(including Hive) access to this information using the metastore service API. 

This hook is registered as post-listener (hive.metastore.event.listeners) and 
DDL operations are captured and send to atlas kafka topic for processing by 
Atlas server.

The following DDL operations are captured:
 * CreateDatabaseEvent
 * DropDatabaseEvent
 * AlterDatabaseEvent
 * CreateTableEvent
 * DropTableEvent
 * AlterTableEvent

 


> Implement Hive Metastore hook for Atlas
> ---
>
> Key: ATLAS-3148
> URL: https://issues.apache.org/jira/browse/ATLAS-3148
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk
>
>
> Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which 
> registers with HiveServer2 process to listen for create/update/delete 
> operations and updates the metadata in Atlas.
> If hive metastore is accessed using other clients like Impala shell, Hue or 
> other JDBC/ODBC client apps there is no way to capture the events in Atlas.
> This Jira will create a new atlas hook for Hive Metastore - stores the 
> metadata for Hive tables and partitions in a relational database, and 
> provides clients (including Hive) access to this information using the 
> metastore service API. 
> This hook is registered as post-listener (*hive.metastore.event.listeners*) 
> and DDL operations are captured and send to atlas kafka topic for processing 
> by Atlas server.
> The following DDL operations are captured:
>  * *CreateDatabaseEvent*
>  * *DropDatabaseEvent*
>  * *AlterDatabaseEvent*
>  * *CreateTableEvent*
>  * *DropTableEvent*
>  * *AlterTableEvent*
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3211) Update message to ATLAS_HOOK from hive-hook to use relationshipAttributes

2019-05-15 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3211:
--
Description: Currently in Hive Hook object id reference attributes are 
added as part of "attributes" and we don't leverage "relationshipAttributes". 
This JIRA will update hive hook to assign all objectid references using 
relationship attributes.  (was: Currently in Hive Hook object if reference 
attributes are added as part of "attributes" and we don't leverage 
relationshipAttributes. This JIRA will update hive hook to assign all objectif 
references using relationship attributes.)

> Update message to ATLAS_HOOK from hive-hook to use relationshipAttributes
> -
>
> Key: ATLAS-3211
> URL: https://issues.apache.org/jira/browse/ATLAS-3211
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> Currently in Hive Hook object id reference attributes are added as part of 
> "attributes" and we don't leverage "relationshipAttributes". This JIRA will 
> update hive hook to assign all objectid references using relationship 
> attributes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3211) Update message to ATLAS_HOOK from hive-hook to use relationshipAttributes

2019-05-15 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3211:
--
Description: Currently in Hive Hook object if reference attributes are 
added as part of "attributes" and we don't leverage relationshipAttributes. 
This JIRA will update hive hook to assign all objectif references using 
relationship attributes.  (was: Currently Hive Hook doesn't uses 
relationshipAttributes to create entities. This jira will update hive hook code 
to use relationships.)

> Update message to ATLAS_HOOK from hive-hook to use relationshipAttributes
> -
>
> Key: ATLAS-3211
> URL: https://issues.apache.org/jira/browse/ATLAS-3211
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> Currently in Hive Hook object if reference attributes are added as part of 
> "attributes" and we don't leverage relationshipAttributes. This JIRA will 
> update hive hook to assign all objectif references using relationship 
> attributes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (ATLAS-3100) Enhance Full Text Search backend implementation to use SOLR request handler.

2019-05-15 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian resolved ATLAS-3100.
---
Resolution: Fixed

> Enhance Full Text Search backend implementation to use SOLR request handler.
> 
>
> Key: ATLAS-3100
> URL: https://issues.apache.org/jira/browse/ATLAS-3100
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: Sridhar
>Assignee: Sridhar
>Priority: Major
> Attachments: Atlas Improvements--Support for Free text search, 
> Auto-Suggestion and Faceting..pdf
>
>
> For full text search, Atlas is using separate collection with name 
> "Full-Text-Search" for text searches. This collection maintains index on a 
> Full Text string that is created using all the attributes of the Entity. This 
> improvement will try to eliminate the need to maintain this collection and 
> use SOLR search more efficiently.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (ATLAS-3120) Release Atlas 2.0.0

2019-05-15 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian resolved ATLAS-3120.
---
Resolution: Fixed

> Release Atlas 2.0.0
> ---
>
> Key: ATLAS-3120
> URL: https://issues.apache.org/jira/browse/ATLAS-3120
> Project: Atlas
>  Issue Type: Task
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0
>
>
> parent task to capture release tasks for Apache Atlas 2.0



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (ATLAS-3181) The URL produced by the swagger interactive API is missing "/api/atlas/"

2019-05-17 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian reassigned ATLAS-3181:
-

Assignee: Carol Drummond

> The URL produced by the swagger interactive API is missing "/api/atlas/"
> 
>
> Key: ATLAS-3181
> URL: https://issues.apache.org/jira/browse/ATLAS-3181
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Carol Drummond
>Assignee: Carol Drummond
>Priority: Minor
> Attachments: ATLAS-3181.patch
>
>
> Swagger isn't building the API url correctly: it's missing the "api/atlas" 
> parts of the URL:
>  
> Here's what it builds:
> [https://localhost:21000/v2/entity/guid/4b2aa848-fb3a-401a-9e05-29a08ab2c846/classifications]
> {quote}curl -X GET --header 'Accept: application/json' 
> '[https://localhost:21000/v2/entity/guid/4b2aa848-fb3a-401a-9e05-29a08ab2c846/classifications]'{quote}
>  
> Here's what works (added "/api/atlas"):
>  
> [http://localhost:21000|http://localhost:21000/]*/api/atlas/*v2/entity/guid/4b2aa848-fb3a-401a-9e05-29a08ab2c846/classifications
>  
> {quote}curl -X GET 
> '[http://localhost:21000/api/atlas/v2/entity/guid/4b2aa848-fb3a-401a-9e05-29a08ab2c846/classifications]'
>  -u user:password{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3260) When importing KAFKA metadata, the following error is reported

2019-06-04 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855810#comment-16855810
 ] 

Sarath Subramanian commented on ATLAS-3260:
---

[~langxia], can you verify the kafka version, it should be kafka_2.11-2.0.0.jar

> When importing KAFKA metadata, the following error is reported
> --
>
> Key: ATLAS-3260
> URL: https://issues.apache.org/jira/browse/ATLAS-3260
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.1.0
>Reporter: He sheng
>Priority: Major
>
> Adding Kafka topic ATLAS_HOOK
> Exception in thread "main" java.lang.NoSuchMethodError: 
> kafka.utils.ZkUtils.getTopicPartitionCount(Ljava/lang/String;)Lscala/Option;
>     at 
> org.apache.atlas.kafka.bridge.KafkaBridge.getTopicEntity(KafkaBridge.java:231)
>     at 
> org.apache.atlas.kafka.bridge.KafkaBridge.createOrUpdateTopic(KafkaBridge.java:196)
>     at 
> org.apache.atlas.kafka.bridge.KafkaBridge.importTopic(KafkaBridge.java:182)
>     at org.apache.atlas.kafka.bridge.KafkaBridge.main(KafkaBridge.java:139)
> Failed to import Kafka Data Model!!!   
> thread
> !moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/drowDown.svg!
> [THred]
> !moz-extension://8260c77a-a03d-49f1-8b6d-70c335e701e7/static/sound.svg!
> 名词线; 丝; 线索; 绪; 细线; 缕; 运作; 丝状物
> 动词缕; 纫
> 定义
> 名词a long, thin strand of cotton, nylon, or other fibers used in sewing or 
> weaving.
> 动词pass a thread through the eye of (a needle) or through the needle and 
> guides of (a sewing machine).
> 同义词
> 名词cotton; filament; fiber; yarn; string; twine
> 动词pass; string; work; ease; push; poke
> 示例
> She sighed and let her hands rest on her lap, her fingers absently playing 
> with a loose *thread* of cotton.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3241) AtlasClientV2Test.java invalid directory

2019-06-04 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855803#comment-16855803
 ] 

Sarath Subramanian commented on ATLAS-3241:
---

[~Zep], Thanks for the pull request. I have merged the changes.

> AtlasClientV2Test.java invalid directory
> 
>
> Key: ATLAS-3241
> URL: https://issues.apache.org/jira/browse/ATLAS-3241
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0, trunk, 2.1.0, 3.0.0
> Environment: All
>Reporter: Diego Marino Monetti
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk, 2.1.0, 3.0.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> AtlasClientV2Test.java should have the following path
> {quote}client/client-v2 /src/test 
> /java/{color:#FF}org/apache/atlas{color}/AtlasClientV2Test.java 
> {quote}
> instead has
> {quote}client/client-v2 
> /src/test/java/{color:#FF}org.apache.atlas{color}/AtlasClientV2Test.
> {quote}
> Bug or, for some strange reason, intentional ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (ATLAS-3241) AtlasClientV2Test.java invalid directory

2019-06-04 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855375#comment-16855375
 ] 

Sarath Subramanian edited comment on ATLAS-3241 at 6/4/19 3:00 PM:
---

Precommit - 
[https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1166/console]


was (Author: sarath.ku...@gmail.com):
Precommit - 
[https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1165/console]

> AtlasClientV2Test.java invalid directory
> 
>
> Key: ATLAS-3241
> URL: https://issues.apache.org/jira/browse/ATLAS-3241
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0, trunk, 2.1.0, 3.0.0
> Environment: All
>Reporter: Diego Marino Monetti
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.0.0, trunk, 2.1.0, 3.0.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> AtlasClientV2Test.java should have the following path
> {quote}client/client-v2 /src/test 
> /java/{color:#FF}org/apache/atlas{color}/AtlasClientV2Test.java 
> {quote}
> instead has
> {quote}client/client-v2 
> /src/test/java/{color:#FF}org.apache.atlas{color}/AtlasClientV2Test.
> {quote}
> Bug or, for some strange reason, intentional ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3153) Support OpenID Connect directly rather than through Knox

2019-05-28 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16850056#comment-16850056
 ] 

Sarath Subramanian commented on ATLAS-3153:
---

[~bolke], like Srikanth mentiioned - " a design doc for the use case and the 
arch. and configs needed for Atlas to consume OpenID asserted credentials? 
Perhaps an interaction diagram to explain how this feature will work, the 
actual flow, and configs needed (for example how the user/group mappings are 
fetched) "

> Support OpenID Connect directly rather than through Knox
> 
>
> Key: ATLAS-3153
> URL: https://issues.apache.org/jira/browse/ATLAS-3153
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Bolke de Bruin
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The current SSO implementation with Apache Knox is limiting SSO 
> interoperability to Apache Knox. Knox uses JWT verification which could 
> easily be extended to allow for direct OpenID Connect support and doesn't 
> require organizations to deploy Knox.
> Required changes:
>  * Pickup bearer token from headers
>  * Improve and standardize redirecting
>  * Optionally: obtain certificates from well_known uri
>  * Optionally: obtain user groups from userinfo endpoint rather than UGI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (ATLAS-3240) Switch all 'http' urls to 'https' in maven settings

2019-05-28 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-3240:
-

 Summary: Switch all 'http' urls to 'https' in maven settings
 Key: ATLAS-3240
 URL: https://issues.apache.org/jira/browse/ATLAS-3240
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 1.2.0, 2.1.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 1.2.0, 2.1.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ATLAS-3153) Support OpenID Connect directly rather than through Knox

2019-05-28 Thread Sarath Subramanian (JIRA)


[ 
https://issues.apache.org/jira/browse/ATLAS-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16850011#comment-16850011
 ] 

Sarath Subramanian commented on ATLAS-3153:
---

[~bolke] can you attach the design doc in the JIRA.

> Support OpenID Connect directly rather than through Knox
> 
>
> Key: ATLAS-3153
> URL: https://issues.apache.org/jira/browse/ATLAS-3153
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Bolke de Bruin
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The current SSO implementation with Apache Knox is limiting SSO 
> interoperability to Apache Knox. Knox uses JWT verification which could 
> easily be extended to allow for direct OpenID Connect support and doesn't 
> require organizations to deploy Knox.
> Required changes:
>  * Pickup bearer token from headers
>  * Improve and standardize redirecting
>  * Optionally: obtain certificates from well_known uri
>  * Optionally: obtain user groups from userinfo endpoint rather than UGI



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (ATLAS-3240) Update 'http' urls to 'https' in maven settings

2019-05-28 Thread Sarath Subramanian (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3240:
--
Summary: Update 'http' urls to 'https' in maven settings  (was: Switch all 
'http' urls to 'https' in maven settings)

> Update 'http' urls to 'https' in maven settings
> ---
>
> Key: ATLAS-3240
> URL: https://issues.apache.org/jira/browse/ATLAS-3240
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.2.0, 2.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Minor
> Fix For: 1.2.0, 2.1.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


<    1   2   3   4   5   6   7   8   9   10   >