Re: Review Request 73329: Correlating Deleted Entities with Lineage

2021-05-04 Thread Ashutosh Mestry via Review Board

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73329/
---

(Updated May 5, 2021, 12:39 a.m.)


Review request for atlas, Radhika Kundam and Sarath Subramanian.


Changes
---

Updates include:
- Modified approach. Now leveraging serializers for adding spool attribute.
- Utilizing msgCreated for within correlations.


Bugs: ATLAS-4152
https://issues.apache.org/jira/browse/ATLAS-4152


Repository: atlas


Description
---

**Background**
As part of ATLAS-4204, HS2 notifications send entity-lineage only (provided the 
poperty is enabled).

When Spooling enabled the order of messages can potentially change. The 
notification messages coming from HS2 and HMS may not be in the same order as 
when they arrived with direct notification.

Problem: 
Consider the sequence of arriving messages:

This is the sequence of messages for Entity 1 (C = create, U = update, D = 
delete, L?x = Lineage of type 'x')
No problem: C1, U1, L1x, L1y, D1
Problem: C1, U1, D1, L1x, L1y

This implementation attempts to handle ths problem mentioned above. If the 
above case is not handled, it will end up creating shell entities, since 
deleted entities are not looked up as part of entity creation.

**Approach**
Used bounded stream approach where an incoming stream of messages is bounded 
with an indicator that it originates from spool. This helps makes localized 
decisions on the incoming stream of messages.

High-level approach:
- Messages when written to the spool are tagged with a timestamp.
- Deleted entities are maintained in a cache.
- Lineage-only message are checked if they refer to a deleted entity.
- If they refer to deleted entity, they are stitched to the one present in the 
cache only if it falls within the threshold.

New: _EntityCorrelationsManager_: Uses message timestamp and cached entity 
qualifiedName-GUID map.
Modifed: _NotificationHookConsumer_ Uses the new class.
New: _HiveDDLLineagePreprocess_ Uses entity-correlation to link to deleted 
entities.


Diffs (updated)
-

  
intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationMessage.java
 810ba97c9 
  notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 9162ac144 
  notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
f7d9668ec 
  notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaMessage.java 
22bd79fdf 
  
notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java
 c45a1da95 
  
notification/src/main/java/org/apache/atlas/notification/AtlasNotificationMessageDeserializer.java
 3264e264c 
  notification/src/main/java/org/apache/atlas/notification/spool/Spooler.java 
2cacaaadc 
  
webapp/src/main/java/org/apache/atlas/notification/EntityCorrelationManager.java
 PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 84cc8d813 
  
webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java
 89568e236 
  
webapp/src/main/java/org/apache/atlas/notification/preprocessor/HiveDdlLineagePreprocessor.java
 PRE-CREATION 
  
webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java
 e69d63e3a 
  
webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java
 608b4a304 


Diff: https://reviews.apache.org/r/73329/diff/4/

Changes: https://reviews.apache.org/r/73329/diff/3-4/


Testing
---

**Functional tests**
Manual verification of scenarios.


Thanks,

Ashutosh Mestry



[jira] [Created] (ATLAS-4274) [Atlas: Glossary] Non matching relation are created via bulk import

2021-05-04 Thread Dharshana M Krishnamoorthy (Jira)
Dharshana M Krishnamoorthy created ATLAS-4274:
-

 Summary: [Atlas: Glossary] Non matching relation are created via 
bulk import
 Key: ATLAS-4274
 URL: https://issues.apache.org/jira/browse/ATLAS-4274
 Project: Atlas
  Issue Type: Bug
Reporter: Dharshana M Krishnamoorthy
 Attachments: Screenshot 2021-05-04 at 3.31.00 PM.png, Screenshot 
2021-05-04 at 3.34.03 PM.png, Screenshot 2021-05-04 at 3.34.21 PM.png, 
Screenshot 2021-05-04 at 3.34.36 PM.png

The related terms provided in the input does not match the relation created via 
import
{code:java}
GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
a_glossary_1,term_1,,,"a_glossary_1:term_2"
a_glossary_1,term_2,"a_glossary_1:term_3",,
a_glossary_1,term_3,,"a_glossary_1:term_1", {code}
!Screenshot 2021-05-04 at 3.31.00 PM.png|width=1973,height=127!

!Screenshot 2021-05-04 at 3.34.03 PM.png|width=1038,height=578!

!Screenshot 2021-05-04 at 3.34.21 PM.png|width=1005,height=563!

!Screenshot 2021-05-04 at 3.34.36 PM.png|width=541,height=303!
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ATLAS-4273) [Atlas: Glossary Term Bulk Import] When there is only 1 term imported via bulk import and if it fails, no proper reason is mentioned in response

2021-05-04 Thread Dharshana M Krishnamoorthy (Jira)
Dharshana M Krishnamoorthy created ATLAS-4273:
-

 Summary: [Atlas: Glossary Term Bulk Import] When there is only 1 
term imported via bulk import and if it fails, no proper reason is mentioned in 
response
 Key: ATLAS-4273
 URL: https://issues.apache.org/jira/browse/ATLAS-4273
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: Dharshana M Krishnamoorthy


The following fails as we do not support @ in term name
{code:java}
GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
glossary_l,term_@_1 {code}
But the failure is
{code:java}
{"errorCode":"ATLAS-409-00-011","errorMessage":"Glossary import failed"} {code}
The code is 409 which mean a conflict but the reason is different



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3887) Consistency and highlighting issues while switching between old and new UI

2021-05-04 Thread Prasad P. Pawar (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-3887:
---
Fix Version/s: 2.2.0
   3.0.0

> Consistency and highlighting issues while switching between old and new UI
> --
>
> Key: ATLAS-3887
> URL: https://issues.apache.org/jira/browse/ATLAS-3887
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Durga Kadam
>Assignee: Prasad P. Pawar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 
> 0001-ATLAS-3887-ATLAS-4242-UI-Consistency-and-highlightin.patch, 
> classification and glossari does not highlight also glossary does not show 
> title when switched to new UI.mkv
>
>
> Description::
> When switched from old to new UI,
>  # Classification and Glossary does not highlight the content, also
>  # Glossary does not show the title. 
>  # On Old UI, glossary tab, click on term category toggle button,
>  on click
>  - button gets toggled to Category
>  - Import/Download glossary button gets disabled.
>  Now switch to New-UI
>  on left side Glossary panel shows Category tree
>  - button dosent toggled to Category (bug)
>  - import/Download Glossary options is enabled (bug)
> PFA video



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-4242) (New UI) Single click of glossary doesn't work after clicking on Atlas logo.

2021-05-04 Thread Prasad P. Pawar (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-4242:
---
Fix Version/s: 2.2.0
   3.0.0

> (New UI) Single click of glossary doesn't work after clicking on Atlas logo.
> 
>
> Key: ATLAS-4242
> URL: https://issues.apache.org/jira/browse/ATLAS-4242
> Project: Atlas
>  Issue Type: Bug
>Reporter: Rahul Kurup
>Assignee: Prasad P. Pawar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
>
> Steps:
>  # Select any one existing glossary and single click it.
>  # Then click on Atlas logo
>  # Then once again single click the same glossary
> You will see that the glossary does not get selected as expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-4240) UI: Some Categories are not shown in the tree that matches the search string of search bar.

2021-05-04 Thread Prasad P. Pawar (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-4240:
---
Fix Version/s: 2.2.0
   3.0.0

> UI: Some Categories are not shown in the tree that matches the search string 
> of search bar.
> ---
>
> Key: ATLAS-4240
> URL: https://issues.apache.org/jira/browse/ATLAS-4240
> Project: Atlas
>  Issue Type: Sub-task
>  Components: atlas-webui
>Reporter: Prasad P. Pawar
>Assignee: Prasad P. Pawar
>Priority: Major
>  Labels: categories
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 
> 0001-ATLAS-3903-UI-Glossary-category-not-reflected-in-the.patch, 
> CategoryList.png, Old-ui_Category_Bug.png, Old-ui_Category_bug_2.png
>
>
> Consider "G6" glossary having "G6_C1" category, and "G6_C1" category having 
> "G6_SubCategory" category (refer image CategoryList.png)
> Old-UI:
> Glossary tab-> switch to category and refresh the page
> Search for a category "G6"
> It doesn't show "G6_SubCategory" in the search (refer to image 
> Old-ui_Category_Bug.png).
> Expand the "G6_C1" category, Api is hit and "G6_SubCategory" is not 
> highlighted (refer to image Old-ui_Category_bug_2.png)
> New-UI (left side search bar):
> Glossary panel-> switch to category
> Search for a category "G6" (left side search bar)
> It doesn't show "G6_SubCategory" in the search
> Expand "G6_C1" category, Api is hit and "G6_SubCategory" is not highlighted



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-4244) Atlas: UI Lineage tab gives Uncaught TypeError

2021-05-04 Thread Prasad P. Pawar (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-4244:
---
Fix Version/s: 2.2.0
   3.0.0

> Atlas: UI Lineage tab gives Uncaught TypeError
> --
>
> Key: ATLAS-4244
> URL: https://issues.apache.org/jira/browse/ATLAS-4244
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Reporter: Prasad P. Pawar
>Assignee: Prasad P. Pawar
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 
> 0001-ATLAS-UI-Lineage-without-displayText-key-fixed.patch, 4244_bug.png
>
>
> 1) The lineage tab gives Uncaught TypeError in the console panel, when 
> lineage without displayText & attribute keys. (refer 4244_bug.png)
> LineageLayoutView.js?bust=1576185031805:208 Uncaught TypeError: Cannot read 
> property 'trunc' of undefined
> at N.d. (LineageLayoutView.js?bust=1576185031805:208)
> at LineageLayoutView.js?bust=1576185031805:222
> at Function.s.each.s.forEach (underscore-min.js?bust=1576185031805:1)
> at N.d.crateLineageRelationshipHashMap 
> (LineageLayoutView.js?bust=1576185031805:221)
> at Object.success (LineageLayoutView.js?bust=1576185031805:147)
> at k (jquery.min.js?bust=1576185031805:2)
> at Object.fireWith [as resolveWith] (jquery.min.js?bust=1576185031805:2)
> at d (jquery.min.js?bust=1576185031805:3)
> at XMLHttpRequest. (jquery.min.js?bust=1576185031805:3)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-4245) New UI: "Download import template" and "Import Glossary term" gets blocked on valid navigation.

2021-05-04 Thread Prasad P. Pawar (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-4245:
---
Fix Version/s: 2.2.0
   3.0.0

> New UI: "Download import template" and "Import Glossary term" gets blocked on 
> valid navigation.
> ---
>
> Key: ATLAS-4245
> URL: https://issues.apache.org/jira/browse/ATLAS-4245
> Project: Atlas
>  Issue Type: Bug
>Reporter: Rahul Kurup
>Assignee: Prasad P. Pawar
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 
> 0001-ATLAS-4245-New-UI-Download-import-template-and-Impor.patch
>
>
> Steps:
>  # In old UI, navigate to Classification tab
>  # Switch to new UI
>  # Scroll down to Glossary section
>  # Click on the ellipsis menu
> When you hover over the "Download import template" and "Import Glossary term" 
> options, you will be blocked from clicking them. This is because after 
> switching from old UI to new UI, the glossary panel shows Category tree 
> instead of Term tree.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-4252) UI (New): Deleting subclassifications results in endless loading

2021-05-04 Thread Prasad P. Pawar (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad P. Pawar updated ATLAS-4252:
---
Fix Version/s: 2.2.0
   3.0.0

> UI (New): Deleting subclassifications results in endless loading
> 
>
> Key: ATLAS-4252
> URL: https://issues.apache.org/jira/browse/ATLAS-4252
> Project: Atlas
>  Issue Type: Bug
>Reporter: Rahul Kurup
>Assignee: Prasad P. Pawar
>Priority: Minor
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 
> 0001-ATLAS-4252-UI-New-Deleting-subclassifications-result.patch
>
>
> Steps:
>  # In new UI, create classification
>  # Create sub-classification for that classification
>  # Create sub-classification for that sub-classification
>  # Start deleting sub-classifications from latest to upwards
> You will see that while deleting, the confirmation prompt for one of the 
> sub-classifications shows endless loading. Only on browser refresh it goes 
> away and we are able to delete it normally.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-4263) KafkaUtils sets invalid dynamic JAAS config

2021-05-04 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338885#comment-17338885
 ] 

ASF subversion and git services commented on ATLAS-4263:


Commit e8c9bd8216e451a3db3afeda562e8986527bc6b4 in atlas's branch 
refs/heads/branch-2.0 from Vlad Glinsky
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e8c9bd8 ]

ATLAS-4263: Fix KafkaUtils to always enclose property values in double-quotes 
(#137)

(cherry picked from commit 21606bdc20e3890ddf9d26555fc71e4175310950)


> KafkaUtils sets invalid dynamic JAAS config
> ---
>
> Key: ATLAS-4263
> URL: https://issues.apache.org/jira/browse/ATLAS-4263
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 2.1.0, 3.0.0
>Reporter: Vladislav Glinskiy
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
>  doesn't always 
> [enclose|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L316]
>  property values in double-quotes, thus, setting invalid dynamic JAAS config 
> for token auth in some cases.
> Faced this issue with Spark Atlas Connector trying to configure Atlas client 
> to use delegation tokens. The following configuration is not handled properly:
> {code:java}
> atlas.jaas.KafkaClient.option.username=30CQ4q1hQMy0dB6X0eXfxQ
> atlas.jaas.KafkaClient.option.password=KdaUQ4FlKWlDxwQrAeFGUVbb6sR0P+zoqOZDZjtIRP1wseXbSbhiTjz3QI9Ur9o4LTYZSv8TE1QqUC4FSwnoTA==
> {code}
> and results in the following error:
> {code:java}
> java.lang.IllegalArgumentException: Value not specified for key 'null' in 
> JAAS config
>   at 
> org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:116)
>   at 
> org.apache.kafka.common.security.JaasConfig.(JaasConfig.java:63)
>   at 
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90)
>   at 
> org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84)
> {code}
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
>  should always enclose property values in double-quotes, since unenclosed 
> digits and '+' sign can not be parsed by Kafka 
> [JaasConfig|https://github.com/apache/kafka/blob/2.0.0/clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java#L116].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-4263) KafkaUtils sets invalid dynamic JAAS config

2021-05-04 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338883#comment-17338883
 ] 

ASF subversion and git services commented on ATLAS-4263:


Commit 21606bdc20e3890ddf9d26555fc71e4175310950 in atlas's branch 
refs/heads/master from Vlad Glinsky
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=21606bd ]

ATLAS-4263: Fix KafkaUtils to always enclose property values in double-quotes 
(#137)



> KafkaUtils sets invalid dynamic JAAS config
> ---
>
> Key: ATLAS-4263
> URL: https://issues.apache.org/jira/browse/ATLAS-4263
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 2.1.0, 3.0.0
>Reporter: Vladislav Glinskiy
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
>  doesn't always 
> [enclose|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L316]
>  property values in double-quotes, thus, setting invalid dynamic JAAS config 
> for token auth in some cases.
> Faced this issue with Spark Atlas Connector trying to configure Atlas client 
> to use delegation tokens. The following configuration is not handled properly:
> {code:java}
> atlas.jaas.KafkaClient.option.username=30CQ4q1hQMy0dB6X0eXfxQ
> atlas.jaas.KafkaClient.option.password=KdaUQ4FlKWlDxwQrAeFGUVbb6sR0P+zoqOZDZjtIRP1wseXbSbhiTjz3QI9Ur9o4LTYZSv8TE1QqUC4FSwnoTA==
> {code}
> and results in the following error:
> {code:java}
> java.lang.IllegalArgumentException: Value not specified for key 'null' in 
> JAAS config
>   at 
> org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:116)
>   at 
> org.apache.kafka.common.security.JaasConfig.(JaasConfig.java:63)
>   at 
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90)
>   at 
> org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84)
> {code}
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
>  should always enclose property values in double-quotes, since unenclosed 
> digits and '+' sign can not be parsed by Kafka 
> [JaasConfig|https://github.com/apache/kafka/blob/2.0.0/clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java#L116].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [atlas] nixonrodrigues merged pull request #137: ATLAS-4263: Fix KafkaUtils to always enclose property values in double-quotes

2021-05-04 Thread GitBox


nixonrodrigues merged pull request #137:
URL: https://github.com/apache/atlas/pull/137


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] nixonrodrigues commented on pull request #137: ATLAS-4263: Fix KafkaUtils to always enclose property values in double-quotes

2021-05-04 Thread GitBox


nixonrodrigues commented on pull request #137:
URL: https://github.com/apache/atlas/pull/137#issuecomment-831821119


   +1 for the PR. Thanks @vladhlinsky 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (ATLAS-4263) KafkaUtils sets invalid dynamic JAAS config

2021-05-04 Thread Jayendra Parab (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338882#comment-17338882
 ] 

Jayendra Parab commented on ATLAS-4263:
---

+1 for the patch

> KafkaUtils sets invalid dynamic JAAS config
> ---
>
> Key: ATLAS-4263
> URL: https://issues.apache.org/jira/browse/ATLAS-4263
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 2.1.0, 3.0.0
>Reporter: Vladislav Glinskiy
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
>  doesn't always 
> [enclose|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L316]
>  property values in double-quotes, thus, setting invalid dynamic JAAS config 
> for token auth in some cases.
> Faced this issue with Spark Atlas Connector trying to configure Atlas client 
> to use delegation tokens. The following configuration is not handled properly:
> {code:java}
> atlas.jaas.KafkaClient.option.username=30CQ4q1hQMy0dB6X0eXfxQ
> atlas.jaas.KafkaClient.option.password=KdaUQ4FlKWlDxwQrAeFGUVbb6sR0P+zoqOZDZjtIRP1wseXbSbhiTjz3QI9Ur9o4LTYZSv8TE1QqUC4FSwnoTA==
> {code}
> and results in the following error:
> {code:java}
> java.lang.IllegalArgumentException: Value not specified for key 'null' in 
> JAAS config
>   at 
> org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:116)
>   at 
> org.apache.kafka.common.security.JaasConfig.(JaasConfig.java:63)
>   at 
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90)
>   at 
> org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84)
> {code}
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
>  should always enclose property values in double-quotes, since unenclosed 
> digits and '+' sign can not be parsed by Kafka 
> [JaasConfig|https://github.com/apache/kafka/blob/2.0.0/clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java#L116].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3817) Could not instantiate implementation: org.janusgraph.diskstorage.es.ElasticSearchIndex

2021-05-04 Thread Arpit Shah (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338876#comment-17338876
 ] 

Arpit Shah commented on ATLAS-3817:
---

hi ..if you able to solve this issue then please let me know

> Could not instantiate implementation: 
> org.janusgraph.diskstorage.es.ElasticSearchIndex
> --
>
> Key: ATLAS-3817
> URL: https://issues.apache.org/jira/browse/ATLAS-3817
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 3.0.0
>Reporter: Gaurav Sehgal
>Priority: Major
>
> Getting this error while starting the atlas server with Elasticsearch.
> {code:java}
> java.lang.IllegalArgumentException: Could not instantiate implementation: 
> org.janusgraph.diskstorage.es.ElasticSearchIndexjava.lang.IllegalArgumentException:
>  Could not instantiate implementation: 
> org.janusgraph.diskstorage.es.ElasticSearchIndex at 
> org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:64)
>  at 
> org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:440) 
> at org.janusgraph.diskstorage.Backend.getIndexes(Backend.java:427) at 
> org.janusgraph.diskstorage.Backend.(Backend.java:150) at 
> org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1359)
>  at 
> org.janusgraph.graphdb.database.StandardJanusGraph.(StandardJanusGraph.java:146)
>  at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161) at 
> org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132) at 
> org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:112) at 
> org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.initJanusGraph(AtlasJanusGraphDatabase.java:182)
>  at 
> org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraphInstance(AtlasJanusGraphDatabase.java:169)
>  at 
> org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraph(AtlasJanusGraphDatabase.java:276)
>  at 
> org.apache.atlas.repository.graph.AtlasGraphProvider.getGraphInstance(AtlasGraphProvider.java:52)
>  at 
> org.apache.atlas.repository.graph.AtlasGraphProvider.retry(AtlasGraphProvider.java:114)
>  at 
> org.apache.atlas.repository.graph.AtlasGraphProvider.get(AtlasGraphProvider.java:102)
>  at 
> org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$35082b34.CGLIB$get$1()
>  at 
> org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$35082b34$$FastClassBySpringCGLIB$$12375d15.invoke()
>  at 
> org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) 
> at 
> org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
>  at 
> org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$35082b34.get()
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498) at 
> org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
>  at 
> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
>  at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1178)
>  at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1072)
>  at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
>  at 
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
>  at 
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
>  at 
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
>  at 
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
>  at 
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
>  at 
> org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
>  at 
> org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1136)
>  at 
>