[jira] [Commented] (ATLAS-3855) Bulk entity tag association and bulk api enhancement
[
https://issues.apache.org/jira/browse/ATLAS-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17154239#comment-17154239
]
ASF subversion and git services commented on ATLAS-3855:
Commit 06cd0cb35a285b3c4a7b0b3c662ce7263848f7b6 in atlas's branch
refs/heads/master from chaitali borole
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=06cd0cb ]
ATLAS-3855 :- Bulk entity tag association and bulk api enhancement,
authorization fix.
Signed-off-by: nixonrodrigues
> Bulk entity tag association and bulk api enhancement
>
>
> Key: ATLAS-3855
> URL: https://issues.apache.org/jira/browse/ATLAS-3855
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 2.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Labels: api-change
> Fix For: 2.1.0
>
>
> API : /api/atlas/v2/entity/bulk/classification
> Body :
> {code:java}
> {"classification":{"typeName":"PII","attributes":{},"propagate":true,"removePropagationsOnEntityDelete":false,"validityPeriods":[]},"entityGuids":["guid1","guid2"]}
> {code}
> when user doesn't have authorisation on either of guid1 or guid2 , the bulk
> entity call fails with 403.
> bulk api _v2/entity/bulk_ to retrieve a list of entities identified by its
> guids.
> This bulk api fails with 403 if some guids belong to entities on which user
> is *unauthorized* and other guids belong to entities on which user is
> *authorized*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3878) Notifications: Improve Memory Usage in Scale Enviroment
[ https://issues.apache.org/jira/browse/ATLAS-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3878: --- Attachment: (was: notifications-memory-fix-v3.patch) > Notifications: Improve Memory Usage in Scale Enviroment > --- > > Key: ATLAS-3878 > URL: https://issues.apache.org/jira/browse/ATLAS-3878 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0, trunk >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3878-Notifications-Improve-Memory-Usage-in-Sca-v2.patch > > > *Background* > As part of entity creation, Atlas sends notifications of different types. > Current implementation, to listeners. Listeners in turn perform specific > tasks. > At a more concrete level, the _EntityAuditListenerV2_ will write audits and > the _NotificationEntityChangeListener_ will send Kafka notifications. > Each of the listeners create notification objects. The notification objects > are large in number and are short lived. > The transient nature of the notification objects causes memory pressure in > scale environment. > *Solution* > Create object pool for notification objects. This way objects can be > reused.and existing design can be kept in tact. This will also offer benefit > of using existing test setup for verification. > *Tests Used* > _Setup_ > Create a test rig that will spawn multiple works that will invoke Atlas' bulk > APIs for entity creation. > Node: 40 workers, 8 GB allocated memory and 40 cores. > _Observation_ > About 40 mins into the exercise, memory pressure builds up causing GC > collects to take longer. This causes ZK timeout and finally Atlas process > crashes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3878) Notifications: Improve Memory Usage in Scale Enviroment
[ https://issues.apache.org/jira/browse/ATLAS-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3878: --- Attachment: ATLAS-3878-Notifications-Improve-Memory-Usage-in-Sca-v2.patch > Notifications: Improve Memory Usage in Scale Enviroment > --- > > Key: ATLAS-3878 > URL: https://issues.apache.org/jira/browse/ATLAS-3878 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0, trunk >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3878-Notifications-Improve-Memory-Usage-in-Sca-v2.patch > > > *Background* > As part of entity creation, Atlas sends notifications of different types. > Current implementation, to listeners. Listeners in turn perform specific > tasks. > At a more concrete level, the _EntityAuditListenerV2_ will write audits and > the _NotificationEntityChangeListener_ will send Kafka notifications. > Each of the listeners create notification objects. The notification objects > are large in number and are short lived. > The transient nature of the notification objects causes memory pressure in > scale environment. > *Solution* > Create object pool for notification objects. This way objects can be > reused.and existing design can be kept in tact. This will also offer benefit > of using existing test setup for verification. > *Tests Used* > _Setup_ > Create a test rig that will spawn multiple works that will invoke Atlas' bulk > APIs for entity creation. > Node: 40 workers, 8 GB allocated memory and 40 cores. > _Observation_ > About 40 mins into the exercise, memory pressure builds up causing GC > collects to take longer. This causes ZK timeout and finally Atlas process > crashes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 72666: Notification: Solution to Memory Build-up
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72666/ --- Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3878 https://issues.apache.org/jira/browse/ATLAS-3878 Repository: atlas Description --- **Background** See JIRA for details. *Analysis* Using memory profiling tools, it was observed that large number of notification objects were created. These stayed in memory and later were promoted to higher generation, thereby taking even longer to be collected. **Approach** Using the fixed-buffer approach to address the problem of creating large number of small objects. New *FixedBufferList* This is an encapsulation over *ArrayList*. During initial allocation, list is populated with default values. Features: - Setting of values to these pre-allocated objects is achieved by first doing a *get* on the element and then assigning values to it. - *toList* fetches the sub-list from the encapsulating list. This uses the state within the class to fetch the right length for the returning array. New *NamedFixedBufferList* Maintains a per-thread *FixedBufferList*. This is necessary since the list is now part class's state. Modified *EntityAuditListenerV2* Uses the new classes. Modifed *EntityNotificationListener* Uses the new classes. **Verification** - Using the test setup, the memory usage was observed over a period of 24 hrs. - Memory usage and object allocation was obvserved using memory profiler. Diffs - intg/src/main/java/org/apache/atlas/utils/FixedBufferList.java PRE-CREATION intg/src/main/java/org/apache/atlas/utils/NamedFixedBufferList.java PRE-CREATION intg/src/test/java/org/apache/atlas/utils/FixedBufferListTest.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java 79527acfa webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java a677b315c Diff: https://reviews.apache.org/r/72666/diff/1/ Testing --- **Unit testing** Unit tests added for the new classes. **Volume testing** Setup: - Node: Threads 40, Core: 40, Allocated Memory: 12 GB - Multiple Kafka queues ingesting data. - Bulk entity creation using custom script ingesting 100M entities. Memory usage stayed between 0 and 5% during the 24 hr period. **Pre-commit** https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/2035/ Thanks, Ashutosh Mestry
[jira] [Updated] (ATLAS-3878) Notifications: Improve Memory Usage in Scale Enviroment
[ https://issues.apache.org/jira/browse/ATLAS-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3878: --- Attachment: notifications-memory-fix-v3.patch > Notifications: Improve Memory Usage in Scale Enviroment > --- > > Key: ATLAS-3878 > URL: https://issues.apache.org/jira/browse/ATLAS-3878 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0, trunk >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: notifications-memory-fix-v3.patch > > > *Background* > As part of entity creation, Atlas sends notifications of different types. > Current implementation, to listeners. Listeners in turn perform specific > tasks. > At a more concrete level, the _EntityAuditListenerV2_ will write audits and > the _NotificationEntityChangeListener_ will send Kafka notifications. > Each of the listeners create notification objects. The notification objects > are large in number and are short lived. > The transient nature of the notification objects causes memory pressure in > scale environment. > *Solution* > Create object pool for notification objects. This way objects can be > reused.and existing design can be kept in tact. This will also offer benefit > of using existing test setup for verification. > *Tests Used* > _Setup_ > Create a test rig that will spawn multiple works that will invoke Atlas' bulk > APIs for entity creation. > Node: 40 workers, 8 GB allocated memory and 40 cores. > _Observation_ > About 40 mins into the exercise, memory pressure builds up causing GC > collects to take longer. This causes ZK timeout and finally Atlas process > crashes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3878) Notifications: Improve Memory Usage in Scale Enviroment
[ https://issues.apache.org/jira/browse/ATLAS-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3878: --- Description: *Background* As part of entity creation, Atlas sends notifications of different types. Current implementation, to listeners. Listeners in turn perform specific tasks. At a more concrete level, the _EntityAuditListenerV2_ will write audits and the _NotificationEntityChangeListener_ will send Kafka notifications. Each of the listeners create notification objects. The notification objects are large in number and are short lived. The transient nature of the notification objects causes memory pressure in scale environment. *Solution* Create object pool for notification objects. This way objects can be reused.and existing design can be kept in tact. This will also offer benefit of using existing test setup for verification. *Tests Used* _Setup_ Create a test rig that will spawn multiple works that will invoke Atlas' bulk APIs for entity creation. Node: 40 workers, 8 GB allocated memory and 40 cores. _Observation_ About 40 mins into the exercise, memory pressure builds up causing GC collects to take longer. This causes ZK timeout and finally Atlas process crashes. was: *Background* As part of entity creation, Atlas sends notifications of different types. Current implementation, to listeners. Listeners in turn perform specific tasks. At a more concrete level, the _EntityAuditListenerV2_ will write audits and the _NotificationEntityChangeListener_ will send Kafka notifications. Each of the listeners create notification objects. The notification objects are large in number and are short lived. The transient nature of the notification objects causes memory pressure in scale environment. *Solution* Create object pool for notification objects. This way objects can be reused.and existing design can be kept in tact. This will also offer benefit of using existing test setup for verification. *Tests Used* _Setup_ Create a test rig that will spawn multiple works that will invoke Atlas' bulk APIs for entity creation. Node: 40 workers, 8 GB allocated memory and 40 cores. **_Observation_ About 40 mins into the exercise, memory pressure builds up causing GC collects to take longer. This causes ZK timeout and finally Atlas process crashes. > Notifications: Improve Memory Usage in Scale Enviroment > --- > > Key: ATLAS-3878 > URL: https://issues.apache.org/jira/browse/ATLAS-3878 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0, trunk >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > > *Background* > As part of entity creation, Atlas sends notifications of different types. > Current implementation, to listeners. Listeners in turn perform specific > tasks. > At a more concrete level, the _EntityAuditListenerV2_ will write audits and > the _NotificationEntityChangeListener_ will send Kafka notifications. > Each of the listeners create notification objects. The notification objects > are large in number and are short lived. > The transient nature of the notification objects causes memory pressure in > scale environment. > *Solution* > Create object pool for notification objects. This way objects can be > reused.and existing design can be kept in tact. This will also offer benefit > of using existing test setup for verification. > *Tests Used* > _Setup_ > Create a test rig that will spawn multiple works that will invoke Atlas' bulk > APIs for entity creation. > Node: 40 workers, 8 GB allocated memory and 40 cores. > _Observation_ > About 40 mins into the exercise, memory pressure builds up causing GC > collects to take longer. This causes ZK timeout and finally Atlas process > crashes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72651: ATLAS-3880 : BasicSearch: Multiple type/tag: Log invalid and allow searching valid type/tag names
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72651/ --- (Updated July 8, 2020, 4:26 p.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3880 https://issues.apache.org/jira/browse/ATLAS-3880 Repository: atlas Description --- **Issue:** If in basic search, request has combination of valid and invalid typename/tags, it throws "errorCode": "ATLAS-400-00-00A" unknown/invalid typeName for the invalid typeName. **WorkAround:** Log invalid typeNames/tagNames and allow searching for valid **Example:** typeName:"valid,invalid" Allow searching entities for 'valid' type. Diffs - repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 04e82186b repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java b7ce97845 webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 140ed9a6e Diff: https://reviews.apache.org/r/72651/diff/2/ Testing (updated) --- Manually tested Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2033 (In progress) Thanks, Pinal Shah
[jira] [Commented] (ATLAS-3844) Ignore relationship attributes while fetching entities in Import Hive bridge.
[ https://issues.apache.org/jira/browse/ATLAS-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153735#comment-17153735 ] ASF subversion and git services commented on ATLAS-3844: Commit 682d16dd19cb72d36af9798811ed701ecdc0dacf in atlas's branch refs/heads/master from Nixon Rodrigues [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=682d16d ] ATLAS-3844 - Ignore relationship attributes while fetching entities in Import Hive bridge. (UT fix) #2 > Ignore relationship attributes while fetching entities in Import Hive bridge. > - > > Key: ATLAS-3844 > URL: https://issues.apache.org/jira/browse/ATLAS-3844 > Project: Atlas > Issue Type: Improvement > Components: hive-integration >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-ATLAS-3844-Ignore-relationship-attributes-while-fetc.patch, > ATLAS-3844-Testcase-Fix-Ignore-relationship-attributes-while-fetc.patch > > > Currently in Import Hive bridge, the hive_db and hive_table entities are > fetched along relationshipAttributes and related Attributes and this > attributes are cleared thereafter. > Instead fetching relationshipAttributes and related Attributes in response, > it is will good to ignoreRelationship to reduce the size of response payload. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3844) Ignore relationship attributes while fetching entities in Import Hive bridge.
[ https://issues.apache.org/jira/browse/ATLAS-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153731#comment-17153731 ] Pinal commented on ATLAS-3844: -- patch looks good, +1 for the patch. > Ignore relationship attributes while fetching entities in Import Hive bridge. > - > > Key: ATLAS-3844 > URL: https://issues.apache.org/jira/browse/ATLAS-3844 > Project: Atlas > Issue Type: Improvement > Components: hive-integration >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-ATLAS-3844-Ignore-relationship-attributes-while-fetc.patch, > ATLAS-3844-Testcase-Fix-Ignore-relationship-attributes-while-fetc.patch > > > Currently in Import Hive bridge, the hive_db and hive_table entities are > fetched along relationshipAttributes and related Attributes and this > attributes are cleared thereafter. > Instead fetching relationshipAttributes and related Attributes in response, > it is will good to ignoreRelationship to reduce the size of response payload. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3844) Ignore relationship attributes while fetching entities in Import Hive bridge.
[ https://issues.apache.org/jira/browse/ATLAS-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153718#comment-17153718 ] Nixon Rodrigues commented on ATLAS-3844: patch for testcase failure in hive-brigde [^ATLAS-3844-Testcase-Fix-Ignore-relationship-attributes-while-fetc.patch] > Ignore relationship attributes while fetching entities in Import Hive bridge. > - > > Key: ATLAS-3844 > URL: https://issues.apache.org/jira/browse/ATLAS-3844 > Project: Atlas > Issue Type: Improvement > Components: hive-integration >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-ATLAS-3844-Ignore-relationship-attributes-while-fetc.patch, > ATLAS-3844-Testcase-Fix-Ignore-relationship-attributes-while-fetc.patch > > > Currently in Import Hive bridge, the hive_db and hive_table entities are > fetched along relationshipAttributes and related Attributes and this > attributes are cleared thereafter. > Instead fetching relationshipAttributes and related Attributes in response, > it is will good to ignoreRelationship to reduce the size of response payload. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3844) Ignore relationship attributes while fetching entities in Import Hive bridge.
[ https://issues.apache.org/jira/browse/ATLAS-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nixon Rodrigues updated ATLAS-3844: --- Attachment: ATLAS-3844-Testcase-Fix-Ignore-relationship-attributes-while-fetc.patch > Ignore relationship attributes while fetching entities in Import Hive bridge. > - > > Key: ATLAS-3844 > URL: https://issues.apache.org/jira/browse/ATLAS-3844 > Project: Atlas > Issue Type: Improvement > Components: hive-integration >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-ATLAS-3844-Ignore-relationship-attributes-while-fetc.patch, > ATLAS-3844-Testcase-Fix-Ignore-relationship-attributes-while-fetc.patch > > > Currently in Import Hive bridge, the hive_db and hive_table entities are > fetched along relationshipAttributes and related Attributes and this > attributes are cleared thereafter. > Instead fetching relationshipAttributes and related Attributes in response, > it is will good to ignoreRelationship to reduce the size of response payload. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72658: ATLAS-3877: Audit API returns the audit information for an unauthorised entity
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72658/#review221157
---
Ship it!
Ship It!
- Sarath Subramanian
On July 8, 2020, 12:31 a.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72658/
> ---
>
> (Updated July 8, 2020, 12:31 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-3877
> https://issues.apache.org/jira/browse/ATLAS-3877
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-3877: Audit Fixed the authorization for audit both in case of non
> purged entity to populate classifcation and purged entity by getting the
> header from HBase Audit
>
>
> Diffs
> -
>
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> 717310daf
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
> bf1629cb3
> webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679
>
>
> Diff: https://reviews.apache.org/r/72658/diff/2/
>
>
> Testing
> ---
>
> #Deleted Entity
> //Allow
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
> \
> -u admin:admin123
>
> Response:
> [{"entityId":"196bf3a9-07c6-4152-82ad-10176c5df5b9","timestamp":1594192518488,"user":"admin","action":"ENTITY_DELETE","details":"Deleted
>
> entity","eventKey":"196bf3a9-07c6-4152-82ad-10176c5df5b9:1594192518488:1:1594192519277","entity":null,"type":null}]
>
> //Deny
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
> \
> -u sid:sid
>
> Response:
> {"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
> perform read entity: guid=196bf3a9-07c6-4152-82ad-10176c5df5b9"}
>
>
> #Purged Entity
> //Allow
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
> \
> -u admin:admin123
>
> Response:
> [{"entityId":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96","timestamp":1594191071761,"user":"admin","action":"ENTITY_PURGE","details":"Purged:
>
> {\"typeName\":\"hive_table\",\"attributes\":{\"owner\":\"admin\",\"createTime\":1594177077000,\"qualifiedName\":\"default.test_july_7_2020_ctas_10@cm\",\"name\":\"test_july_7_2020_ctas_10\"},\"guid\":\"def375bd-9bad-42f3-9fa7-9edc1c7d8d96\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"DELETED\",\"version\":0,\"classifications\":[{\"typeName\":\"NOT_IMP\",\"entityGuid\":\"5f52ee87-85c8-46ac-ad38-e0e5be935bb2\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"PARENT\",\"entityGuid\":\"73cc5730-494b-4313-8747-9ccf8a1b0227\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false}],\"meanings\":[],\"proxy\":false}","eventKey":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96:1594191071761:3:1594191103634","entity":nu
ll,"type":null}]
>
> //Deny
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
> \
> -u sid:sid
>
> Response:
> {"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
> perform read entity audit: guid=def375bd-9bad-42f3-9fa7-9edc1c7d8d96"}
>
>
> Thanks,
>
> Sidharth Mishra
>
>
[jira] [Commented] (ATLAS-3819) Modernize client side technology stack
[ https://issues.apache.org/jira/browse/ATLAS-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153673#comment-17153673 ] Csaba Koncz commented on ATLAS-3819: Oh, that is just the inverse of the express-webpack-dev-server integration :) [https://webpack.js.org/guides/development/#using-webpack-dev-middleware] Will try that too, the only question is how can I test the result of the production build (the content of "dist" folder)? Shall I start webpack-dev-server with mode:"production" and then can we assume that it serves the same content that it otherwise generates to "dist"? > Modernize client side technology stack > -- > > Key: ATLAS-3819 > URL: https://issues.apache.org/jira/browse/ATLAS-3819 > Project: Atlas > Issue Type: Improvement > Components: atlas-webui >Affects Versions: 3.0.0 >Reporter: Csaba Koncz >Priority: Major > Attachments: image-2020-07-04-16-42-10-643.png > > > While RequireJS has its merits and advantages, it might be worth considering > replacing it with another technology. > A bundle loader like Webpack would open up a number of possibilities: > - use of ES6 modules, which immediately improves tooling support > - use TypeScript, which improves tooling support even more > - a manageable code base would enable refactoring of the JavaScript code so > that no logic happens in the module initialization code, > which in turn would make it possible to write tests for the JavaScript logic. > As a POC I created a NodeJS project that transforms the current dashboardv2 > code into Webpack bundles and demonstrates > that the UI is able to work without RequireJS. > Right now the JavaScript sources and index.html are used unchanged, > but the next step would be to start converting the source to ES6 modules. > Here is the code: https://github.com/csabakoncz/atlas-newui > The most important file there is `webpack.config.js` that takes over the role > of RequireJS.config. > The other file of interest might be `src/index.ts` which demonstrates the > ability to use TypeScript code even now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3877) Purged entity audit throws 404 Entity not found error
[ https://issues.apache.org/jira/browse/ATLAS-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153659#comment-17153659 ] ASF subversion and git services commented on ATLAS-3877: Commit 16ced369e741a00cd31da4d6fd5d4fa3dc05ec1a in atlas's branch refs/heads/master from sidmishra [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=16ced36 ] ATLAS-3877: fix for error in retrieving audit for purged entity Signed-off-by: Madhan Neethiraj > Purged entity audit throws 404 Entity not found error > - > > Key: ATLAS-3877 > URL: https://issues.apache.org/jira/browse/ATLAS-3877 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sharmadha S >Assignee: Sidharth Kumar Mishra >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3877.patch, rc2.png > > > 1. Create an hdfs_path entity. > 2. Delete the entity using DELETE: /api/atlas/v2/entity/guid/. The > entity is soft deleted > 3. Purge the entity using PUT : /api/atlas/admin/purge with body [""] > 4. Go to admin -> Administration -> Audits > 5. Click on the GUID of purged entity. > The purged entity details window opens but Atlas throws a 404 error "Given > entity GUID is invalid/not found" . Attached screenshot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3877) Purged entity audit throws 404 Entity not found error
[ https://issues.apache.org/jira/browse/ATLAS-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153660#comment-17153660 ] ASF subversion and git services commented on ATLAS-3877: Commit da3551938deceaf714f80ec8c0927161646cb8bb in atlas's branch refs/heads/branch-2.0 from sidmishra [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=da35519 ] ATLAS-3877: fix for error in retrieving audit for purged entity Signed-off-by: Madhan Neethiraj (cherry picked from commit 16ced369e741a00cd31da4d6fd5d4fa3dc05ec1a) > Purged entity audit throws 404 Entity not found error > - > > Key: ATLAS-3877 > URL: https://issues.apache.org/jira/browse/ATLAS-3877 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sharmadha S >Assignee: Sidharth Kumar Mishra >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3877.patch, rc2.png > > > 1. Create an hdfs_path entity. > 2. Delete the entity using DELETE: /api/atlas/v2/entity/guid/. The > entity is soft deleted > 3. Purge the entity using PUT : /api/atlas/admin/purge with body [""] > 4. Go to admin -> Administration -> Audits > 5. Click on the GUID of purged entity. > The purged entity details window opens but Atlas throws a 404 error "Given > entity GUID is invalid/not found" . Attached screenshot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72658: ATLAS-3877: Audit API returns the audit information for an unauthorised entity
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72658/#review221155
---
Ship it!
Ship It!
- Madhan Neethiraj
On July 8, 2020, 7:31 a.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72658/
> ---
>
> (Updated July 8, 2020, 7:31 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-3877
> https://issues.apache.org/jira/browse/ATLAS-3877
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-3877: Audit Fixed the authorization for audit both in case of non
> purged entity to populate classifcation and purged entity by getting the
> header from HBase Audit
>
>
> Diffs
> -
>
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> 717310daf
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
> bf1629cb3
> webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679
>
>
> Diff: https://reviews.apache.org/r/72658/diff/2/
>
>
> Testing
> ---
>
> #Deleted Entity
> //Allow
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
> \
> -u admin:admin123
>
> Response:
> [{"entityId":"196bf3a9-07c6-4152-82ad-10176c5df5b9","timestamp":1594192518488,"user":"admin","action":"ENTITY_DELETE","details":"Deleted
>
> entity","eventKey":"196bf3a9-07c6-4152-82ad-10176c5df5b9:1594192518488:1:1594192519277","entity":null,"type":null}]
>
> //Deny
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
> \
> -u sid:sid
>
> Response:
> {"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
> perform read entity: guid=196bf3a9-07c6-4152-82ad-10176c5df5b9"}
>
>
> #Purged Entity
> //Allow
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
> \
> -u admin:admin123
>
> Response:
> [{"entityId":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96","timestamp":1594191071761,"user":"admin","action":"ENTITY_PURGE","details":"Purged:
>
> {\"typeName\":\"hive_table\",\"attributes\":{\"owner\":\"admin\",\"createTime\":1594177077000,\"qualifiedName\":\"default.test_july_7_2020_ctas_10@cm\",\"name\":\"test_july_7_2020_ctas_10\"},\"guid\":\"def375bd-9bad-42f3-9fa7-9edc1c7d8d96\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"DELETED\",\"version\":0,\"classifications\":[{\"typeName\":\"NOT_IMP\",\"entityGuid\":\"5f52ee87-85c8-46ac-ad38-e0e5be935bb2\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"PARENT\",\"entityGuid\":\"73cc5730-494b-4313-8747-9ccf8a1b0227\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false}],\"meanings\":[],\"proxy\":false}","eventKey":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96:1594191071761:3:1594191103634","entity":nu
ll,"type":null}]
>
> //Deny
> curl --location --request GET
> 'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
> \
> -u sid:sid
>
> Response:
> {"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
> perform read entity audit: guid=def375bd-9bad-42f3-9fa7-9edc1c7d8d96"}
>
>
> Thanks,
>
> Sidharth Mishra
>
>
[jira] [Updated] (ATLAS-3885) import-hive.sh: Hive entities with Ozone location created by import-hive.sh creates hdfs_path entity instead of ozone in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Bonte updated ATLAS-3885: Description: import-hive.sh when attempts to import hive entities with Ozone location expected is that ozone_key should get created as path entity, but script creates hdfs_path entity instead of ozone_key in Atlas. Steps to reproduce # Disable HMS Hook and Hive Hook # Create external table with ozone location # Run import-hive.sh The location entity is created as an hdfs_path entity instead of ozone_key. was: import-hive.sh when attempts to import hive entities with Ozone location expected is that ozone_key should get created as path entity, but script creates hdfs_path entity instead of ozone_key in Atlas. Steps to reproduce # Disable HMS Hook and Hive Hook # Create external table with ozone location # Run import-hive.sh The location entity is created as an hdfs_path entity instead of ozone_key. > import-hive.sh: Hive entities with Ozone location created by import-hive.sh > creates hdfs_path entity instead of ozone in Atlas > -- > > Key: ATLAS-3885 > URL: https://issues.apache.org/jira/browse/ATLAS-3885 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Nikhil Bonte >Assignee: Nikhil Bonte >Priority: Major > Labels: ozone > > import-hive.sh when attempts to import hive entities with Ozone location > expected is that ozone_key should get created as path entity, but script > creates hdfs_path entity instead of ozone_key in Atlas. > > Steps to reproduce > # Disable HMS Hook and Hive Hook > # Create external table with ozone location > # Run import-hive.sh > The location entity is created as an hdfs_path entity instead of ozone_key. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3885) import-hive.sh: Hive entities with Ozone location created by import-hive.sh creates hdfs_path entity instead of ozone in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Bonte updated ATLAS-3885: Description: import-hive.sh when attempts to import hive entities with Ozone location expected is that ozone_key should get created as path entity, but script creates hdfs_path entity instead of ozone_key in Atlas. Steps to reproduce # Disable HMS Hook and Hive Hook # Create external table with ozone location # Run import-hive.sh The location entity is created as an hdfs_path entity instead of ozone_key. was: import-hive.sh when attempts to import hive entities with Ozone location expected is that ozone_key will get created as path entity, but script creates hdfs_path entity instead of ozone_key in Atlas. Steps to reproduce # Disable HMS Hook and Hive Hook # Create external table with ozone location # Run import-hive.sh The location entity is created as an hdfs_path entity instead of ozone_key. > import-hive.sh: Hive entities with Ozone location created by import-hive.sh > creates hdfs_path entity instead of ozone in Atlas > -- > > Key: ATLAS-3885 > URL: https://issues.apache.org/jira/browse/ATLAS-3885 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Nikhil Bonte >Assignee: Nikhil Bonte >Priority: Major > Labels: ozone > > import-hive.sh when attempts to import hive entities with Ozone location > expected is that ozone_key should get created as path entity, but script > creates hdfs_path entity instead of ozone_key in Atlas. > > Steps to reproduce > # Disable HMS Hook and Hive Hook > # Create external table with ozone location > # Run import-hive.sh > > The location entity is created as an hdfs_path entity instead of ozone_key. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3885) import-hive.sh: Hive entities with Ozone location created by import-hive.sh creates hdfs_path entity instead of ozone in Atlas
Nikhil Bonte created ATLAS-3885: --- Summary: import-hive.sh: Hive entities with Ozone location created by import-hive.sh creates hdfs_path entity instead of ozone in Atlas Key: ATLAS-3885 URL: https://issues.apache.org/jira/browse/ATLAS-3885 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.0.0 Reporter: Nikhil Bonte Assignee: Nikhil Bonte import-hive.sh when attempts to import hive entities with Ozone location expected is that ozone_key will get created as path entity, but script creates hdfs_path entity instead of ozone_key in Atlas. Steps to reproduce # Disable HMS Hook and Hive Hook # Create external table with ozone location # Run import-hive.sh The location entity is created as an hdfs_path entity instead of ozone_key. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [atlas] dggs123 closed pull request #103: Entities search powered directly through es
dggs123 closed pull request #103: URL: https://github.com/apache/atlas/pull/103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [atlas] dggs123 opened a new pull request #103: Entities search powered directly through es
dggs123 opened a new pull request #103: URL: https://github.com/apache/atlas/pull/103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3882: --- Affects Version/s: 2.0.0 > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3882.patch, Screen Shot 2020-07-07 at 3.18.20 > PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3882: --- Component/s: atlas-webui > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3882.patch, Screen Shot 2020-07-07 at 3.18.20 > PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153517#comment-17153517 ] ASF subversion and git services commented on ATLAS-3882: Commit d604085efe18963f39020e62cd03a5e4fcb20b2b in atlas's branch refs/heads/master from Keval Bhatt [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=d604085 ] ATLAS-3882:- Classic UI: Loader not disappearing while assigning category to term (cherry picked from commit a43b49c33f93ef2819b747caed7a9baae91d09a4) > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3882.patch, Screen Shot 2020-07-07 at 3.18.20 > PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153516#comment-17153516 ] ASF subversion and git services commented on ATLAS-3882: Commit a43b49c33f93ef2819b747caed7a9baae91d09a4 in atlas's branch refs/heads/branch-2.0 from Keval Bhatt [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a43b49c ] ATLAS-3882:- Classic UI: Loader not disappearing while assigning category to term > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3882.patch, Screen Shot 2020-07-07 at 3.18.20 > PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153515#comment-17153515 ] Nixon Rodrigues commented on ATLAS-3882: +1 for the patch, thanks [~kevalbhatt] > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3882.patch, Screen Shot 2020-07-07 at 3.18.20 > PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3884) Unable to access Apache Atlas UI http://localhost:21000
[
https://issues.apache.org/jira/browse/ATLAS-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153509#comment-17153509
]
Keval Bhatt commented on ATLAS-3884:
[~Vidyats]
Can you export these two variables and try to restart atlas.
{code:java}
export MANAGE_LOCAL_HBASE=True
export MANAGE_LOCAL_SOLR=True
{code}
> Unable to access Apache Atlas UI http://localhost:21000
> ---
>
> Key: ATLAS-3884
> URL: https://issues.apache.org/jira/browse/ATLAS-3884
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 2.0.0
> Environment: RHEL
>Reporter: Vidya
>Priority: Major
>
> Hi,I have installed and ran Apache Atlas with embedded Apache HBase & Apache
> Solr.After bin/atlas_start.py command I see apache atlas started message and
> I could grep Atlas,Hbase and Solr process.However,when I do `curl -u
> admin:admin http://localhost:21000/api/atlas/admin/version`, I am getting
> below error
> ```
>
>
>
> Error 503
>
>
> HTTP ERROR: 503
> Problem accessing /api/atlas/admin/version. Reason:
> Service Unavailable
> http://eclipse.org/jetty";>Powered by Jetty://
> 9.3.14.v20161028
>
>
> ```
> I am getting below error in application logs.
> ```
> Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at
> localhost:2181: cluster not found/not ready
> at
> org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)
> at
> org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)
> at org.janusgraph.diskstorage.solr.Solr6Index.(Solr6Index.java:218)
> ... 101 more
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (ATLAS-3884) Unable to access Apache Atlas UI http://localhost:21000
[
https://issues.apache.org/jira/browse/ATLAS-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153509#comment-17153509
]
Keval Bhatt edited comment on ATLAS-3884 at 7/8/20, 11:06 AM:
--
[~Vidyats]
Can you export these two variables and restart atlas.
{code:java}
export MANAGE_LOCAL_HBASE=True
export MANAGE_LOCAL_SOLR=True
{code}
was (Author: kevalbhatt18):
[~Vidyats]
Can you export these two variables and try to restart atlas.
{code:java}
export MANAGE_LOCAL_HBASE=True
export MANAGE_LOCAL_SOLR=True
{code}
> Unable to access Apache Atlas UI http://localhost:21000
> ---
>
> Key: ATLAS-3884
> URL: https://issues.apache.org/jira/browse/ATLAS-3884
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 2.0.0
> Environment: RHEL
>Reporter: Vidya
>Priority: Major
>
> Hi,I have installed and ran Apache Atlas with embedded Apache HBase & Apache
> Solr.After bin/atlas_start.py command I see apache atlas started message and
> I could grep Atlas,Hbase and Solr process.However,when I do `curl -u
> admin:admin http://localhost:21000/api/atlas/admin/version`, I am getting
> below error
> ```
>
>
>
> Error 503
>
>
> HTTP ERROR: 503
> Problem accessing /api/atlas/admin/version. Reason:
> Service Unavailable
> http://eclipse.org/jetty";>Powered by Jetty://
> 9.3.14.v20161028
>
>
> ```
> I am getting below error in application logs.
> ```
> Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at
> localhost:2181: cluster not found/not ready
> at
> org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)
> at
> org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)
> at org.janusgraph.diskstorage.solr.Solr6Index.(Solr6Index.java:218)
> ... 101 more
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-3884) Unable to access Apache Atlas UI http://localhost:21000
Vidya created ATLAS-3884: Summary: Unable to access Apache Atlas UI http://localhost:21000 Key: ATLAS-3884 URL: https://issues.apache.org/jira/browse/ATLAS-3884 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 2.0.0 Environment: RHEL Reporter: Vidya Hi,I have installed and ran Apache Atlas with embedded Apache HBase & Apache Solr.After bin/atlas_start.py command I see apache atlas started message and I could grep Atlas,Hbase and Solr process.However,when I do `curl -u admin:admin http://localhost:21000/api/atlas/admin/version`, I am getting below error ``` Error 503 HTTP ERROR: 503 Problem accessing /api/atlas/admin/version. Reason: Service Unavailable http://eclipse.org/jetty";>Powered by Jetty:// 9.3.14.v20161028 ``` I am getting below error in application logs. ``` Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at localhost:2181: cluster not found/not ready at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141) at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383) at org.janusgraph.diskstorage.solr.Solr6Index.(Solr6Index.java:218) ... 101 more ``` -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3883) Upgrade jackson-databind to version 2.10.0
[ https://issues.apache.org/jira/browse/ATLAS-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nixon Rodrigues resolved ATLAS-3883. Resolution: Duplicate > Upgrade jackson-databind to version 2.10.0 > -- > > Key: ATLAS-3883 > URL: https://issues.apache.org/jira/browse/ATLAS-3883 > Project: Atlas > Issue Type: Bug >Reporter: Nixon Rodrigues >Priority: Major > > Upgrade to jackson-databind to version 2.10.0 > > com.fasterxml.jackson.core > jackson-databind > 2.10.0 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3883) Upgrade jackson-databind to version 2.10.0
[ https://issues.apache.org/jira/browse/ATLAS-3883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153497#comment-17153497 ] Nixon Rodrigues commented on ATLAS-3883: Fix handled in ATLAS-3452 Update jackson.databind.version to 2.10.0 > Upgrade jackson-databind to version 2.10.0 > -- > > Key: ATLAS-3883 > URL: https://issues.apache.org/jira/browse/ATLAS-3883 > Project: Atlas > Issue Type: Bug >Reporter: Nixon Rodrigues >Priority: Major > > Upgrade to jackson-databind to version 2.10.0 > > com.fasterxml.jackson.core > jackson-databind > 2.10.0 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3883) Upgrade jackson-databind to version 2.10.0
Nixon Rodrigues created ATLAS-3883: -- Summary: Upgrade jackson-databind to version 2.10.0 Key: ATLAS-3883 URL: https://issues.apache.org/jira/browse/ATLAS-3883 Project: Atlas Issue Type: Bug Reporter: Nixon Rodrigues Upgrade to jackson-databind to version 2.10.0 com.fasterxml.jackson.core jackson-databind 2.10.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3882: --- Attachment: ATLAS-3882.patch > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3882.patch, Screen Shot 2020-07-07 at 3.18.20 > PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3882) Classic UI: Loader not disappearing while assigning category to term
[ https://issues.apache.org/jira/browse/ATLAS-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3882: --- Summary: Classic UI: Loader not disappearing while assigning category to term (was: Classic UI: Loader not disappearing whie assigning category to term ) > Classic UI: Loader not disappearing while assigning category to term > - > > Key: ATLAS-3882 > URL: https://issues.apache.org/jira/browse/ATLAS-3882 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: Screen Shot 2020-07-07 at 3.18.20 PM.png > > > Steps to reproduce this issue. > > # Go to classic UI > # Click on Glossary tab > # Create term if not exist and click on it > # Click on + button of Categories: from the righthand side view of the term > detail page. > # The popup will be visible with loader. > > !Screen Shot 2020-07-07 at 3.18.20 PM.png|width=579,height=248! -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72658: ATLAS-3877: Audit API returns the audit information for an unauthorised entity
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72658/
---
(Updated July 8, 2020, 7:31 a.m.)
Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
Subramanian.
Bugs: ATLAS-3877
https://issues.apache.org/jira/browse/ATLAS-3877
Repository: atlas
Description
---
ATLAS-3877: Audit Fixed the authorization for audit both in case of non purged
entity to populate classifcation and purged entity by getting the header from
HBase Audit
Diffs (updated)
-
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
717310daf
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
bf1629cb3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679
Diff: https://reviews.apache.org/r/72658/diff/2/
Changes: https://reviews.apache.org/r/72658/diff/1-2/
Testing
---
#Deleted Entity
//Allow
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
\
-u admin:admin123
Response:
[{"entityId":"196bf3a9-07c6-4152-82ad-10176c5df5b9","timestamp":1594192518488,"user":"admin","action":"ENTITY_DELETE","details":"Deleted
entity","eventKey":"196bf3a9-07c6-4152-82ad-10176c5df5b9:1594192518488:1:1594192519277","entity":null,"type":null}]
//Deny
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
\
-u sid:sid
Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
perform read entity: guid=196bf3a9-07c6-4152-82ad-10176c5df5b9"}
#Purged Entity
//Allow
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
\
-u admin:admin123
Response:
[{"entityId":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96","timestamp":1594191071761,"user":"admin","action":"ENTITY_PURGE","details":"Purged:
{\"typeName\":\"hive_table\",\"attributes\":{\"owner\":\"admin\",\"createTime\":1594177077000,\"qualifiedName\":\"default.test_july_7_2020_ctas_10@cm\",\"name\":\"test_july_7_2020_ctas_10\"},\"guid\":\"def375bd-9bad-42f3-9fa7-9edc1c7d8d96\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"DELETED\",\"version\":0,\"classifications\":[{\"typeName\":\"NOT_IMP\",\"entityGuid\":\"5f52ee87-85c8-46ac-ad38-e0e5be935bb2\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"PARENT\",\"entityGuid\":\"73cc5730-494b-4313-8747-9ccf8a1b0227\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false}],\"meanings\":[],\"proxy\":false}","eventKey":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96:1594191071761:3:1594191103634","entity":null
,"type":null}]
//Deny
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
\
-u sid:sid
Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
perform read entity audit: guid=def375bd-9bad-42f3-9fa7-9edc1c7d8d96"}
Thanks,
Sidharth Mishra
Review Request 72658: ATLAS-3877: Audit API returns the audit information for an unauthorised entity
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72658/
---
Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath
Subramanian.
Bugs: ATLAS-3877
https://issues.apache.org/jira/browse/ATLAS-3877
Repository: atlas
Description
---
ATLAS-3877: Audit Fixed the authorization for audit both in case of non purged
entity to populate classifcation and purged entity by getting the header from
HBase Audit
Diffs
-
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
717310daf
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
bf1629cb3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679
Diff: https://reviews.apache.org/r/72658/diff/1/
Testing
---
#Deleted Entity
//Allow
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
\
-u admin:admin123
Response:
[{"entityId":"196bf3a9-07c6-4152-82ad-10176c5df5b9","timestamp":1594192518488,"user":"admin","action":"ENTITY_DELETE","details":"Deleted
entity","eventKey":"196bf3a9-07c6-4152-82ad-10176c5df5b9:1594192518488:1:1594192519277","entity":null,"type":null}]
//Deny
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1&auditAction=ENTITY_DELETE'
\
-u sid:sid
Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
perform read entity: guid=196bf3a9-07c6-4152-82ad-10176c5df5b9"}
#Purged Entity
//Allow
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
\
-u admin:admin123
Response:
[{"entityId":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96","timestamp":1594191071761,"user":"admin","action":"ENTITY_PURGE","details":"Purged:
{\"typeName\":\"hive_table\",\"attributes\":{\"owner\":\"admin\",\"createTime\":1594177077000,\"qualifiedName\":\"default.test_july_7_2020_ctas_10@cm\",\"name\":\"test_july_7_2020_ctas_10\"},\"guid\":\"def375bd-9bad-42f3-9fa7-9edc1c7d8d96\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"DELETED\",\"version\":0,\"classifications\":[{\"typeName\":\"NOT_IMP\",\"entityGuid\":\"5f52ee87-85c8-46ac-ad38-e0e5be935bb2\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"PARENT\",\"entityGuid\":\"73cc5730-494b-4313-8747-9ccf8a1b0227\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false}],\"meanings\":[],\"proxy\":false}","eventKey":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96:1594191071761:3:1594191103634","entity":null
,"type":null}]
//Deny
curl --location --request GET
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1&auditAction=ENTITY_PURGE'
\
-u sid:sid
Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to
perform read entity audit: guid=def375bd-9bad-42f3-9fa7-9edc1c7d8d96"}
Thanks,
Sidharth Mishra
Re: Review Request 72648: ATLAS-3841 Response Headers: Code refactoring
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72648/ --- (Updated July 8, 2020, 7:07 a.m.) Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- Addressed review comments Bugs: ATLAS-3841 https://issues.apache.org/jira/browse/ATLAS-3841 Repository: atlas Description --- Currently headers setting code is scattered at different places. In this patch, code refactoring is done to set response headers from AtlasHeaderFilter.java Currently js/ html files come under the unauthorized urls. Therefore, these files do not have headers which are applied by default by spring security. To overcome this problem, In this patch we are applying all the spring security headers to these unauthorized urls explicitly. We are doing this by mapping all the urls (authorized + unauthorized) to go through one servlet filter which is AtlasHeaderFilter. In case of authorized urls, spring by default set some of the headers. To avoid overwriting of these headers in authorized urls, those headers are skipped in the AtlasHeaderFilter. Also this patch provides flexibility to select file formats to be excluded from applying response headers. Diffs (updated) - distro/src/conf/atlas-application.properties e06e74a intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 2c007ca webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java d9b1c82 webapp/src/main/java/org/apache/atlas/web/filters/AtlasCSRFPreventionFilter.java df3fce6 webapp/src/main/java/org/apache/atlas/web/filters/AtlasDelegatingAuthenticationEntryPoint.java c629a7e webapp/src/main/java/org/apache/atlas/web/filters/AtlasHeaderFilter.java fa7218c webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java 1944a9f webapp/src/main/java/org/apache/atlas/web/filters/HeadersUtil.java 1f8845d webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationFailureHandler.java a117df6 webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationSuccessHandler.java e7a5d66 webapp/src/main/java/org/apache/atlas/web/security/AtlasSecurityConfig.java e74a9e9 webapp/src/main/webapp/WEB-INF/web.xml 2595a15 Diff: https://reviews.apache.org/r/72648/diff/2/ Changes: https://reviews.apache.org/r/72648/diff/1-2/ Testing --- PreCommit: https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2015/console Basic testing done for authorized as well as unauthorized urls, checked whether all the response headers are setting or not. Similar testing done on the knox sso cluster. Also tested from Curl calls. Thanks, Mandar Ambawane
