[jira] [Created] (ATLAS-4335) Hook Notifications through Rest Interface
Deep Singh created ATLAS-4335: - Summary: Hook Notifications through Rest Interface Key: ATLAS-4335 URL: https://issues.apache.org/jira/browse/ATLAS-4335 Project: Atlas Issue Type: New Feature Components: atlas-core Reporter: Deep Singh Assignee: Deep Singh In Data-lake <-> Workload-cluster scenario, versions of the services available on clusters are not always the same. Especially when Kafka versions are different on the clusters, many a time the Kafka client (on workload-cluster) becomes incompatible with the Kafka broker (on data-lake). This leads to failure in the delivery of Atlas Hook notification as the mechanism is dependent on the Kafka client's capability to deliver messages. To resolve such an issue there is no other way but to make Kafka client compatible with the broker by upgrading or degrading the client or the broker. Often this is not straight-fwd and the easiest thing to do. We need another mechanism independent of the Kafka client to deliver hook notifications to the Atlas consumer. There are various REST API available today to perform CURD on Atlas entities. Those APIs process notifications in real-time as they arrive, since event processing is slow and time-consuming these APIs cannot give the throughput which hooks require. We need another REST interface that is lightweight and does not process messages as they arrive instead, it could dump the messages to the local kafka. Since Kafka will be available locally in the cluster, the interface will never face the problem of version mismatch of client-broker, as faced by service hooks running on a different cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4183) web.xml requires external dtd resource
[ https://issues.apache.org/jira/browse/ATLAS-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4183. --- Resolution: Fixed > web.xml requires external dtd resource > -- > > Key: ATLAS-4183 > URL: https://issues.apache.org/jira/browse/ATLAS-4183 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > We have Servlet 2.3 deployment descriptor (web.xml), which follows J2EE 1.3 > DTDs schema. > Therefore it requires dtd resource hosted at > http://java.sun.com/dtd/web-app_2_3.dtd > In setups that are behind firewalls with limited access to external > resources, the Atlas server fails to come up without the external dtd > resource. > This can be fixed by hosting dtd resource locally or upgrading the deployment > descriptor. > We can upgrade it to Servlet 2.5 deployment descriptor which is Java EE 5 XML > schema and it does not require external dtd resource for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4236) [Atlas: Audits][Intermittent] There were 2 audits for an entity update action one with differential enabled and one was without differential within a few milliseconds
[
https://issues.apache.org/jira/browse/ATLAS-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-4236.
---
Resolution: Cannot Reproduce
> [Atlas: Audits][Intermittent] There were 2 audits for an entity update action
> one with differential enabled and one was without differential within a few
> milliseconds
> --
>
> Key: ATLAS-4236
> URL: https://issues.apache.org/jira/browse/ATLAS-4236
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> There was an entity update action and there were 2 audits created, for same
> action one with differential enabled and the other was without differential
> audit enabled within a few milli seconds
> The cluster on which this occurred is still alive, but this was not
> reproduce-able later
> Even if we suspect a config update in atlas and a restart, it would talk
> atleast a few minutes for the config to take effect and the next audit should
> be without differential audit enabled. But this was within a few milli
> seconds.
> [https://quasar-nealzi-1.quasar-nealzi.root.hwx.site:31443/api/atlas/v2/entity/fa12e741-5e5f-45f8-bdd5-239b766e1d76/audit]
> {code:java}
> [
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262937842,
> "user": "hrt_qa",
> "action": "CUSTOM_ATTRIBUTE_UPDATE",
> "details": "Updated custom attribute:
> {\"typeName\":\"entity_type_ozsue\",\"attributes\":{\"type_short_min\":-32768,\"type_bigint_rand\":-5,\"type_bigdecimal_rand\":-5.0,\"type_byte_max\":127,\"type_set\":[\"1\",\"2\",\"3\",\"4\",\"5\"],\"type_short_max\":32767,\"type_bool_true\":true,\"type_str\":\"new_attrib_value\",\"type_double_rand\":-5.0,\"type_float_rand\":-5.0,\"type_date\":1494183548574,\"type_bool_false\":false,\"type_bigdecimal_max\":1.7976931348623157E+308,\"type_bigint_max\":9223372036854775807,\"type_short_rand\":5,\"type_double_max\":1.7976931348623157E308,\"type_bigdecimal_min\":5E-324,\"type_byte_rand\":5,\"type_double_min\":4.9E-324,\"type_int_max\":2147483647,\"type_long_max\":9223372036854775807,\"type_float_min\":300.003,\"type_int_rand\":-5,\"name\":\"audit_entity_rubmo\",\"type_arr_list\":[\"1\",\"2\",\"3\",\"4\",\"5\"],\"type_bigint_min\":-9223372036854775808,\"type_int_min\":9000,\"type_float_max\":3.4E38,\"type_long_min\":-9223372036854775808,\"type_byte_min\":-128,\"type_long_rand\":-5},\"guid\":\"fa12e741-5e5f-45f8-bdd5-239b766e1d76\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"ACTIVE\",\"createdBy\":\"hrt_qa\",\"updatedBy\":\"hrt_qa\",\"createTime\":1617262927193,\"updateTime\":1617262933323,\"version\":0,\"classifications\":[],\"customAttributes\":{\"nseqt\":\"test
> value 3\",\"rojut\":\"test value 1\",\"deyhx\":\"test value
> 2\"},\"labels\":[\"label_cc\",\"label_3\",\"label_01\",\"label_b\",\"label_2\",\"label_a\",\"label_1\"],\"proxy\":false}",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262937842:0:1617262937851",
> "entity": null,
> "type": null
> },
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262933323,
> "user": "hrt_qa",
> "action": "CLASSIFICATION_DELETE",
> "details": "Deleted classification: tag_type_gnpfh",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262933323:0:1617262933349",
> "entity": null,
> "type": null
> },
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262933183,
> "user": "hrt_qa",
> "action": "CLASSIFICATION_UPDATE",
> "details": "Updated classification:
> {\"typeName\":\"tag_type_gnpfh\",\"attributes\":{\"name\":\"val1\",\"id\":1},\"entityGuid\":\"fa12e741-5e5f-45f8-bdd5-239b766e1d76\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262933183:0:1617262933197",
> "entity": null,
> "type": null
> },
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262932983,
> "user": "hrt_qa",
> "action": "CLASSIFICATION_ADD",
> "details": "Added classification:
> {\"typeName\":\"tag_type_gnpfh\",\"attributes\":{},\"entityGuid\":\"fa12e741-5e5f-45f8-bdd5-239b766e1d76\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262932983:0:1617262932994",
> "entity": null,
> "type": null
> },
> {
>
[jira] [Commented] (ATLAS-4236) [Atlas: Audits][Intermittent] There were 2 audits for an entity update action one with differential enabled and one was without differential within a few milliseconds
[
https://issues.apache.org/jira/browse/ATLAS-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319674#comment-17319674
]
Deep Singh commented on ATLAS-4236:
---
Cannot reproduce this one. Closing it now. Dharshana M Krishnamoorthy plz
reopen if you observe it again, with steps to repro.
> [Atlas: Audits][Intermittent] There were 2 audits for an entity update action
> one with differential enabled and one was without differential within a few
> milliseconds
> --
>
> Key: ATLAS-4236
> URL: https://issues.apache.org/jira/browse/ATLAS-4236
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> There was an entity update action and there were 2 audits created, for same
> action one with differential enabled and the other was without differential
> audit enabled within a few milli seconds
> The cluster on which this occurred is still alive, but this was not
> reproduce-able later
> Even if we suspect a config update in atlas and a restart, it would talk
> atleast a few minutes for the config to take effect and the next audit should
> be without differential audit enabled. But this was within a few milli
> seconds.
> [https://quasar-nealzi-1.quasar-nealzi.root.hwx.site:31443/api/atlas/v2/entity/fa12e741-5e5f-45f8-bdd5-239b766e1d76/audit]
> {code:java}
> [
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262937842,
> "user": "hrt_qa",
> "action": "CUSTOM_ATTRIBUTE_UPDATE",
> "details": "Updated custom attribute:
> {\"typeName\":\"entity_type_ozsue\",\"attributes\":{\"type_short_min\":-32768,\"type_bigint_rand\":-5,\"type_bigdecimal_rand\":-5.0,\"type_byte_max\":127,\"type_set\":[\"1\",\"2\",\"3\",\"4\",\"5\"],\"type_short_max\":32767,\"type_bool_true\":true,\"type_str\":\"new_attrib_value\",\"type_double_rand\":-5.0,\"type_float_rand\":-5.0,\"type_date\":1494183548574,\"type_bool_false\":false,\"type_bigdecimal_max\":1.7976931348623157E+308,\"type_bigint_max\":9223372036854775807,\"type_short_rand\":5,\"type_double_max\":1.7976931348623157E308,\"type_bigdecimal_min\":5E-324,\"type_byte_rand\":5,\"type_double_min\":4.9E-324,\"type_int_max\":2147483647,\"type_long_max\":9223372036854775807,\"type_float_min\":300.003,\"type_int_rand\":-5,\"name\":\"audit_entity_rubmo\",\"type_arr_list\":[\"1\",\"2\",\"3\",\"4\",\"5\"],\"type_bigint_min\":-9223372036854775808,\"type_int_min\":9000,\"type_float_max\":3.4E38,\"type_long_min\":-9223372036854775808,\"type_byte_min\":-128,\"type_long_rand\":-5},\"guid\":\"fa12e741-5e5f-45f8-bdd5-239b766e1d76\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"ACTIVE\",\"createdBy\":\"hrt_qa\",\"updatedBy\":\"hrt_qa\",\"createTime\":1617262927193,\"updateTime\":1617262933323,\"version\":0,\"classifications\":[],\"customAttributes\":{\"nseqt\":\"test
> value 3\",\"rojut\":\"test value 1\",\"deyhx\":\"test value
> 2\"},\"labels\":[\"label_cc\",\"label_3\",\"label_01\",\"label_b\",\"label_2\",\"label_a\",\"label_1\"],\"proxy\":false}",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262937842:0:1617262937851",
> "entity": null,
> "type": null
> },
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262933323,
> "user": "hrt_qa",
> "action": "CLASSIFICATION_DELETE",
> "details": "Deleted classification: tag_type_gnpfh",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262933323:0:1617262933349",
> "entity": null,
> "type": null
> },
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262933183,
> "user": "hrt_qa",
> "action": "CLASSIFICATION_UPDATE",
> "details": "Updated classification:
> {\"typeName\":\"tag_type_gnpfh\",\"attributes\":{\"name\":\"val1\",\"id\":1},\"entityGuid\":\"fa12e741-5e5f-45f8-bdd5-239b766e1d76\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}",
> "eventKey":
> "fa12e741-5e5f-45f8-bdd5-239b766e1d76:1617262933183:0:1617262933197",
> "entity": null,
> "type": null
> },
> {
> "entityId": "fa12e741-5e5f-45f8-bdd5-239b766e1d76",
> "timestamp": 1617262932983,
> "user": "hrt_qa",
> "action": "CLASSIFICATION_ADD",
> "details": "Added classification:
> {\"typeName\":\"tag_type_gnpfh\",\"attributes\":{},\"entityGuid\":\"fa12e741-5e5f-45f8-bdd5-239b766e1d76\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}",
> "eventKey":
>
[jira] [Resolved] (ATLAS-4234) [Atlas: Audits] When one of the attributes of a classification is updated, all the attributes are getting displayed in the latest audit
[
https://issues.apache.org/jira/browse/ATLAS-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-4234.
---
Resolution: Invalid
> [Atlas: Audits] When one of the attributes of a classification is updated,
> all the attributes are getting displayed in the latest audit
> ---
>
> Key: ATLAS-4234
> URL: https://issues.apache.org/jira/browse/ATLAS-4234
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> Classification has 2 attributes: *name* and *id*
> *Scenario:*
> Setting both the values initially creates a *classification update* audit
> Eg:
> {code:java}
> {'name': 'val1', 'id': 1} {code}
> audit for the same:
> {code:java}
> u'details': u'Updated classification:
> {"typeName":"tag_type_kgmxl","attributes":{"name":"val1","id":1},"entityGuid":"93160c69-30f4-4fdf-a99b-8228a61d5130","entityStatus":"ACTIVE","propagate":true,"removePropagationsOnEntityDelete":false}',
> {code}
> Now update only one of the attributes
> {code:java}
> {'name': 'new_val'} {code}
> We expect the latest audit should contain only the details of the updated
> attribute , but it has both. Please refer below
> {code:java}
> u'details': u'Updated classification:
> {"typeName":"tag_type_kgmxl","attributes":{"name":"new_val","id":1},"entityGuid":"93160c69-30f4-4fdf-a99b-8228a61d5130","entityStatus":"ACTIVE","propagate":true,"removePropagationsOnEntityDelete":false}'
> {code}
> *Expectation:*
> {code:java}
> u'details': u'Updated classification:
> {"typeName":"tag_type_kgmxl","attributes":{"name":"new_val"},"entityGuid":"93160c69-30f4-4fdf-a99b-8228a61d5130","entityStatus":"ACTIVE","propagate":true,"removePropagationsOnEntityDelete":false}',
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-4234) [Atlas: Audits] When one of the attributes of a classification is updated, all the attributes are getting displayed in the latest audit
[
https://issues.apache.org/jira/browse/ATLAS-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319645#comment-17319645
]
Deep Singh commented on ATLAS-4234:
---
The expectation is invalid. It would suggest the name attribute was changed to
new_val and the id attribute was dropped. Which is not the case.
> [Atlas: Audits] When one of the attributes of a classification is updated,
> all the attributes are getting displayed in the latest audit
> ---
>
> Key: ATLAS-4234
> URL: https://issues.apache.org/jira/browse/ATLAS-4234
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> Classification has 2 attributes: *name* and *id*
> *Scenario:*
> Setting both the values initially creates a *classification update* audit
> Eg:
> {code:java}
> {'name': 'val1', 'id': 1} {code}
> audit for the same:
> {code:java}
> u'details': u'Updated classification:
> {"typeName":"tag_type_kgmxl","attributes":{"name":"val1","id":1},"entityGuid":"93160c69-30f4-4fdf-a99b-8228a61d5130","entityStatus":"ACTIVE","propagate":true,"removePropagationsOnEntityDelete":false}',
> {code}
> Now update only one of the attributes
> {code:java}
> {'name': 'new_val'} {code}
> We expect the latest audit should contain only the details of the updated
> attribute , but it has both. Please refer below
> {code:java}
> u'details': u'Updated classification:
> {"typeName":"tag_type_kgmxl","attributes":{"name":"new_val","id":1},"entityGuid":"93160c69-30f4-4fdf-a99b-8228a61d5130","entityStatus":"ACTIVE","propagate":true,"removePropagationsOnEntityDelete":false}'
> {code}
> *Expectation:*
> {code:java}
> u'details': u'Updated classification:
> {"typeName":"tag_type_kgmxl","attributes":{"name":"new_val"},"entityGuid":"93160c69-30f4-4fdf-a99b-8228a61d5130","entityStatus":"ACTIVE","propagate":true,"removePropagationsOnEntityDelete":false}',
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4183) web.xml requires external dtd resource
Deep Singh created ATLAS-4183: - Summary: web.xml requires external dtd resource Key: ATLAS-4183 URL: https://issues.apache.org/jira/browse/ATLAS-4183 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Deep Singh Assignee: Deep Singh We have Servlet 2.3 deployment descriptor (web.xml), which follows J2EE 1.3 DTDs schema. Therefore it requires dtd resource hosted at http://java.sun.com/dtd/web-app_2_3.dtd In setups that are behind firewalls with limited access to external resources, the Atlas server fails to come up without the external dtd resource. This can be fixed by hosting dtd resource locally or upgrading the deployment descriptor. We can upgrade it to Servlet 2.5 deployment descriptor which is Java EE 5 XML schema and it does not require external dtd resource for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4137) Entity Audit V1 api broken
[ https://issues.apache.org/jira/browse/ATLAS-4137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4137. --- Resolution: Fixed > Entity Audit V1 api broken > -- > > Key: ATLAS-4137 > URL: https://issues.apache.org/jira/browse/ATLAS-4137 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > > Due to the recent introduction of action type Business-attribute-update and > Custom-attribute-update, Entity Audits V1 api is broken. > Problem is in the conversion of EntityAuditEventV2 to EntityAuditEvent (V1 > event). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4137) Entity Audit V1 api broken
[ https://issues.apache.org/jira/browse/ATLAS-4137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh reassigned ATLAS-4137: - Assignee: Deep Singh > Entity Audit V1 api broken > -- > > Key: ATLAS-4137 > URL: https://issues.apache.org/jira/browse/ATLAS-4137 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > > Due to the recent introduction of action type Business-attribute-update and > Custom-attribute-update, Entity Audits V1 api is broken. > Problem is in the conversion of EntityAuditEventV2 to EntityAuditEvent (V1 > event). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4137) Entity Audit V1 api broken
Deep Singh created ATLAS-4137: - Summary: Entity Audit V1 api broken Key: ATLAS-4137 URL: https://issues.apache.org/jira/browse/ATLAS-4137 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Deep Singh Due to the recent introduction of action type Business-attribute-update and Custom-attribute-update, Entity Audits V1 api is broken. Problem is in the conversion of EntityAuditEventV2 to EntityAuditEvent (V1 event). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4136) Export Service: NPE if Options Explicitly set to NULL
[
https://issues.apache.org/jira/browse/ATLAS-4136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17278977#comment-17278977
]
Deep Singh commented on ATLAS-4136:
---
+1 for the patch. LGTM
> Export Service: NPE if Options Explicitly set to NULL
> -
>
> Key: ATLAS-4136
> URL: https://issues.apache.org/jira/browse/ATLAS-4136
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments:
> ATLAS-4136-Export-Service-NPE-if-Options-Explicitly-.patch
>
>
> *Steps to Duplicate*
> # Create an export request by explicitly setting options to null. E.g.
> {code:java}
> {
> "itemsToExport": [{
> "typeName": "hive_db",
> "uniqueAttributes": {
> "qualifiedName": "abcd@cm"
> }
> }],
> "options": null
> }{code}
> _Expected results:_ Export should proceed.
> _Actual results_: Export fails with NPE.
> Root cause: Changes introduced by changes made for ATLAS-4068.
> _Workaround_: Not having _options_ key addresses the problem.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4127) Reduce number of Hive table calls in Atlas Hook
Deep Singh created ATLAS-4127: - Summary: Reduce number of Hive table calls in Atlas Hook Key: ATLAS-4127 URL: https://issues.apache.org/jira/browse/ATLAS-4127 Project: Atlas Issue Type: Bug Reporter: Deep Singh In Atlas Hive bridge, while calculating processQualifiedName, all partitions are processed. For each partition, a hive call is made to get table details. Since multiple partitions can belong to the same table, there is a scope for improvement by avoiding repeated hive calls for the same table information. This improvement becomes crucial particularly in the case where there are several partitions for a single table. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4122) Advanced Search: Fix for within clause with Double Quote Values
[ https://issues.apache.org/jira/browse/ATLAS-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17277527#comment-17277527 ] Deep Singh commented on ATLAS-4122: --- +1 for the patch. > Advanced Search: Fix for within clause with Double Quote Values > --- > > Key: ATLAS-4122 > URL: https://issues.apache.org/jira/browse/ATLAS-4122 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk, 2.1.0 > > Attachments: > ATLAS-4122-Advanced-Search-Literals-with-double-quot.patch > > > *Steps to Duplicate* > # In 'Advance Search' fire a query: _hive_db where name = ["Sales", > "Reporting"]_ > _Expected results:_ With appropriate data being present, the query should > return the correct results. > _Actual results:_ Query is not recognized as valid query and no results are > returned. > Root cause: The change is attributed to the implementation of ATLAS-2932, > where script execution was replaced with traversal. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4110) Result of DSL query inconsistent with basic query result
[
https://issues.apache.org/jira/browse/ATLAS-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-4110.
---
Resolution: Fixed
> Result of DSL query inconsistent with basic query result
>
>
> Key: ATLAS-4110
> URL: https://issues.apache.org/jira/browse/ATLAS-4110
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
>
> Create two tables through the beeline prompt, test1, and test2. Set owners
> for both as "abc". Set table property "comment"="Hello!" on one of the
> tables(test2).
> Basic query to /api/atlas/v2/search/basic
> --data-binary '{"entityFilters":{"condition":"AND", "criterion": [
> {"attributeName":"comment", "operator":"neq", "attributeValue":"abc"}
> , {"attributeName":"owner", "operator":"eq", "attributeValue":"abc"}]},
> "tagFilters":null, "attributes":"["comment", "qualifiedName"], "limit":25,
> "offset":0, "typeName":"hive_table", "classification":null, "termName":null}
> responds with both the tables.
> DSL query (owner="abc" and comment!="abc" & typeName=hive_table).
> /api/atlas/v2/search/dsl?limit=25=0=owner%3D%22abc%22%20and%20comment!%3D%22abc%22=hive_table
> responds with just one table having comment(test2)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4110) Result of DSL query inconsistent with basic query result
[
https://issues.apache.org/jira/browse/ATLAS-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh updated ATLAS-4110:
--
Summary: Result of DSL query inconsistent with basic query result (was:
DSL query inconsistent with basic query)
> Result of DSL query inconsistent with basic query result
>
>
> Key: ATLAS-4110
> URL: https://issues.apache.org/jira/browse/ATLAS-4110
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
>
> Create two tables through the beeline prompt, test1, and test2. Set owners
> for both as "abc". Set table property "comment"="Hello!" on one of the
> tables(test2).
> Basic query to /api/atlas/v2/search/basic
> --data-binary '{"entityFilters":{"condition":"AND", "criterion": [
> {"attributeName":"comment", "operator":"neq", "attributeValue":"abc"}
> , {"attributeName":"owner", "operator":"eq", "attributeValue":"abc"}]},
> "tagFilters":null, "attributes":"["comment", "qualifiedName"], "limit":25,
> "offset":0, "typeName":"hive_table", "classification":null, "termName":null}
> responds with both the tables.
> DSL query (owner="abc" and comment!="abc" & typeName=hive_table).
> /api/atlas/v2/search/dsl?limit=25=0=owner%3D%22abc%22%20and%20comment!%3D%22abc%22=hive_table
> responds with just one table having comment(test2)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4110) DSL query inconsistent with basic query
Deep Singh created ATLAS-4110:
-
Summary: DSL query inconsistent with basic query
Key: ATLAS-4110
URL: https://issues.apache.org/jira/browse/ATLAS-4110
Project: Atlas
Issue Type: Bug
Components: atlas-core
Reporter: Deep Singh
Assignee: Deep Singh
Create two tables through the beeline prompt, test1, and test2. Set owners for
both as "abc". Set table property "comment"="Hello!" on one of the
tables(test2).
Basic query to /api/atlas/v2/search/basic
--data-binary '{"entityFilters":{"condition":"AND", "criterion": [
{"attributeName":"comment", "operator":"neq", "attributeValue":"abc"}
, {"attributeName":"owner", "operator":"eq", "attributeValue":"abc"}]},
"tagFilters":null, "attributes":"["comment", "qualifiedName"], "limit":25,
"offset":0, "typeName":"hive_table", "classification":null, "termName":null}
responds with both the tables.
DSL query (owner="abc" and comment!="abc" & typeName=hive_table).
/api/atlas/v2/search/dsl?limit=25=0=owner%3D%22abc%22%20and%20comment!%3D%22abc%22=hive_table
responds with just one table having comment(test2)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ATLAS-4085) The result page in dsl does not contain links to the result when "__guid" is not a part of the query
[
https://issues.apache.org/jira/browse/ATLAS-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-4085.
---
Resolution: Not A Problem
The link can be formed by UI only if GUID information is present. UI code makes
the best effort to create links with available information. If in the query you
are specifying not to include GUID, there is no way UI can formulate links.
Closing it as `Not A Problem`
> The result page in dsl does not contain links to the result when "__guid" is
> not a part of the query
>
>
> Key: ATLAS-4085
> URL: https://issues.apache.org/jira/browse/ATLAS-4085
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> In Atlas advanced search, enter the following query, the results for this
> page are in plain_text format
> {code:java}
> from hive_table where name LIKE "table_abc_*" select name, owner
> {code}
> Now, enter the following query
> {code:java}
> from hive_table where name LIKE "table_abc_*" select name, __guid
> {code}
> Same tables are filtered but now the results are click-able on UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4085) The result page in dsl does not contain links to the result when "__guid" is not a part of the query
[
https://issues.apache.org/jira/browse/ATLAS-4085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh reassigned ATLAS-4085:
-
Assignee: Deep Singh
> The result page in dsl does not contain links to the result when "__guid" is
> not a part of the query
>
>
> Key: ATLAS-4085
> URL: https://issues.apache.org/jira/browse/ATLAS-4085
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> In Atlas advanced search, enter the following query, the results for this
> page are in plain_text format
> {code:java}
> from hive_table where name LIKE "table_abc_*" select name, owner
> {code}
> Now, enter the following query
> {code:java}
> from hive_table where name LIKE "table_abc_*" select name, __guid
> {code}
> Same tables are filtered but now the results are click-able on UI
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4094) Sorting functionality on Atlas entity audit api
Deep Singh created ATLAS-4094: - Summary: Sorting functionality on Atlas entity audit api Key: ATLAS-4094 URL: https://issues.apache.org/jira/browse/ATLAS-4094 Project: Atlas Issue Type: Improvement Components: atlas-core Reporter: Deep Singh Assignee: Deep Singh We are implementing Atlas audits UI in Data Catalog. For that, we need the ability to sort the events by given field. Right now there is no such support in the API. We'll need the ability to sort the events based on timestamp, action or user. The API is documented here - http://atlas.apache.org/api/v2/resource_EntityREST.html#resource_EntityREST_getAuditEvents_GET -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4076. --- Resolution: Fixed > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4067) In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
[ https://issues.apache.org/jira/browse/ATLAS-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4067. --- Fix Version/s: 2.1.0 Resolution: Fixed > In Entity Audits tab, show action as "user-defined properties > added/updated/deleted" instead of plain 'Entity Updated' > -- > > Key: ATLAS-4067 > URL: https://issues.apache.org/jira/browse/ATLAS-4067 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Umesh Padashetty >Assignee: Deep Singh >Priority: Major > Fix For: 2.1.0 > > Attachments: Screenshot 2020-12-09 at 1.03.44 AM.png, Screenshot > 2020-12-09 at 12.57.59 AM.png > > > Currently in the entity audits tab, we show actions as below: > * Business Attribute(s) Updated > * Classification Added > * Label(s) Added > !Screenshot 2020-12-09 at 1.03.44 AM.png|width=1017,height=440! > But for user-defined properties alone, we show "Entity Updated" for all the > udp actions. > !Screenshot 2020-12-09 at 12.57.59 AM.png|width=1028,height=292! > This makes it very difficult to find out from the audits tab in which "Entity > Update" action the udp was updated for an entity. > Though the user can expand the operation and find out what happened with udp, > this is not very intuitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248148#comment-17248148 ] Deep Singh commented on ATLAS-4076: --- Analysis: = After detecting relationship and creating relationship edge, the propagated enteties(classifications) are notified to entityChangeListner through entityChangeNotifier. However details of the propagated enteties are not passed directly to notifier, but through request context (buffered into addedPropagation list). After processing every edge, AtlasRelationshipStore manager sends notification to entityChangeListner, which simply gets all the items in request context buffer list. In this issue, Hive sends and event which has multiple relationships, and only one relationship has propagated entities, but due to multiple notifications(which is correct) same buffer list is processed multipletimes (which is wrong). > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh updated ATLAS-4076: -- External issue URL: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/263/console > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh updated ATLAS-4076: -- External issue URL: (was: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/263/console) > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
Deep Singh created ATLAS-4076: - Summary: Creating an asset from existing asset adds duplicate Atlas audit entries Key: ATLAS-4076 URL: https://issues.apache.org/jira/browse/ATLAS-4076 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.1.0 Reporter: Deep Singh Assignee: Deep Singh Steps : Go to beeline in hive. Create a new asset which is created from an existing asset, using the query : create table test_web_site as select * from web_site_five; Observe that web_site_five contains two managed tags which should be propagated to the newly created table , i.e : test_web_site Observe the audit tab for 'test_web_site' table in atlas contains multiple entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4067) In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
[ https://issues.apache.org/jira/browse/ATLAS-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh reassigned ATLAS-4067: - Assignee: Deep Singh > In Entity Audits tab, show action as "user-defined properties > added/updated/deleted" instead of plain 'Entity Updated' > -- > > Key: ATLAS-4067 > URL: https://issues.apache.org/jira/browse/ATLAS-4067 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Umesh Padashetty >Assignee: Deep Singh >Priority: Major > Attachments: Screenshot 2020-12-09 at 1.03.44 AM.png, Screenshot > 2020-12-09 at 12.57.59 AM.png > > > Currently in the entity audits tab, we show actions as below: > * Business Attribute(s) Updated > * Classification Added > * Label(s) Added > !Screenshot 2020-12-09 at 1.03.44 AM.png|width=1017,height=440! > But for user-defined properties alone, we show "Entity Updated" for all the > udp actions. > !Screenshot 2020-12-09 at 12.57.59 AM.png|width=1028,height=292! > This makes it very difficult to find out from the audits tab in which "Entity > Update" action the udp was updated for an entity. > Though the user can expand the operation and find out what happened with udp, > this is not very intuitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4059) Rename Configuration flag for importing internal Kafka topics
[ https://issues.apache.org/jira/browse/ATLAS-4059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4059. --- Resolution: Fixed > Rename Configuration flag for importing internal Kafka topics > - > > Key: ATLAS-4059 > URL: https://issues.apache.org/jira/browse/ATLAS-4059 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.2.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > 'atlas.hook.kafka.import.internal.topics' flag was introduced through > ATLAS-4043 to enable import of Kafka internal topics. Rename it to > 'atlas.kafka.bridge.enable.internal.topics.import' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4049) Issue with few /admin APIs when hit on PASSIVE Atlas instance in HA mode
[
https://issues.apache.org/jira/browse/ATLAS-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-4049.
---
Resolution: Fixed
> Issue with few /admin APIs when hit on PASSIVE Atlas instance in HA mode
>
>
> Key: ATLAS-4049
> URL: https://issues.apache.org/jira/browse/ATLAS-4049
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Umesh Padashetty
>Assignee: Deep Singh
>Priority: Critical
>
> Atlas supports the following types of /admin APIs
> # /version
> # /audits
> # /purge
> # /metrics
> # /status
> # /export
> # /import
> # /patches
> # /checkstate
> # /activeSearches/\{id}
> # /activeSearches
> # /audit/\{auditGuid}/details
> # /expimp/audit
> # /server/\{serverName}
> # /importfile
> # /stack
> # /session
> Observing issues when these APIs are hit on PASSIVE Atlas instance in HA mode
> Following APIs do not have any issue, when hit on ACTIVE or PASSIVE instance,
> they produce the same output
> # /version
> # /status
> # /session
> # /stack
> # /patches
> # /activeSearches
> # /activeSearches/\{id}
> # /checkstate
> But the following APIs have issue, when hit on the PASSIVE instance, they
> either produce partial information, or they throw an ERROR.
> * /audits
> ** Error:
> {code:java}
> {"errorCode":"ATLAS-400-00-00A","errorMessage":"__AtlasAuditEntry:
> Unknown/invalid typename"} {code}
> * /purge
> ** Error:
> {code:java}
> {"errorCode":"ATLAS-400-00-014","errorMessage":"Type ENTITY with name
> aws_s3_v2_object does not exist"} {code}
> * /export (I understand this is by design, so can be ignored?)
> ** Error:
> {code:java}
> 2020-11-24 18:06:59,250 ERROR - [etp402695541-112:] ~ URL not supported in HA
> mode: /api/atlas/admin/export (ActiveServerFilter:120) 2020-11-24
> 18:06:59,252 INFO - [etp402695541-112:] ~ Not active. Redirecting to
> https://quasar-jfuidh-4.quasar-jfuidh.root.hwx.site:31443/api/atlas/admin/export
> (ActiveServerFilter:172) {code}
> * /import (I understand this is by design, so can be ignored?)
> ** Error:
> {code:java}
> 2020-11-24 18:07:22,702 ERROR - [etp402695541-118:] ~ URL not supported in HA
> mode: /api/atlas/admin/import (ActiveServerFilter:120) 2020-11-24
> 18:07:22,704 INFO - [etp402695541-118:] ~ Not active. Redirecting to
> https://quasar-jfuidh-4.quasar-jfuidh.root.hwx.site:31443/api/atlas/admin/import
> (ActiveServerFilter:172) {code}
> * /importfile (I understand this is by design, so can be ignored?)
> ** Error:
> {code:java}
> 2020-11-24 18:07:22,702 ERROR - [etp402695541-118:] ~ URL not supported in HA
> mode: /api/atlas/admin/import (ActiveServerFilter:120) 2020-11-24
> 18:07:22,704 INFO - [etp402695541-118:] ~ Not active. Redirecting to
> https://quasar-jfuidh-4.quasar-jfuidh.root.hwx.site:31443/api/atlas/admin/import
> (ActiveServerFilter:172) {code}
> * /audit/\{auditGuid}/details
> ** Error:
> {code:java}
> { "errorCode": "ATLAS-404-00-001", "errorMessage": "Given typename
> __AtlasAuditEntry was invalid" } {code}
> * /server/\{serverName}
> ** Error:
> {code:java}
> There was an error processing your request. It has been logged (ID
> a52953c92b609fb8). 2020-11-24 18:05:32,259 ERROR - [etp402695541-70 -
> 47fbcd4a-e547-4bba-b12b-af018e33a893:] ~ Error handling a request:
> a52953c92b609fb8 (ExceptionMapperUtil:32) java.lang.NullPointerException
> {code}
> * /metrics
> ** Error:
> {code:java}
> Returns only partial information {code}
> * /expimp/audit
> ** Error:
> {code:java}
> { "errorCode": "ATLAS-400-00-00A", "errorMessage": "__ExportImportAuditEntry:
> Unknown/invalid typename" } {code}
> The case when an /admin API is hit on the PASSIVE instance, we should either
> return a proper response, or we should redirect the request to the ACTIVE
> instance. Providing partial information/error seems to be incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4059) Rename Configuration flag for importing internal Kafka topics
Deep Singh created ATLAS-4059: - Summary: Rename Configuration flag for importing internal Kafka topics Key: ATLAS-4059 URL: https://issues.apache.org/jira/browse/ATLAS-4059 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.2.0 Reporter: Deep Singh Assignee: Deep Singh Fix For: 3.0.0, 2.2.0 'atlas.hook.kafka.import.internal.topics' flag was introduced through ATLAS-4043 to enable import of Kafka internal topics. Rename it to 'atlas.kafka.bridge.enable.internal.topics.import' -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4049) Issue with few /admin APIs when hit on PASSIVE Atlas instance in HA mode
[
https://issues.apache.org/jira/browse/ATLAS-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh reassigned ATLAS-4049:
-
Assignee: Deep Singh
> Issue with few /admin APIs when hit on PASSIVE Atlas instance in HA mode
>
>
> Key: ATLAS-4049
> URL: https://issues.apache.org/jira/browse/ATLAS-4049
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Umesh Padashetty
>Assignee: Deep Singh
>Priority: Critical
>
> Atlas supports the following types of /admin APIs
> # /version
> # /audits
> # /purge
> # /metrics
> # /status
> # /export
> # /import
> # /patches
> # /checkstate
> # /activeSearches/\{id}
> # /activeSearches
> # /audit/\{auditGuid}/details
> # /expimp/audit
> # /server/\{serverName}
> # /importfile
> # /stack
> # /session
> Observing issues when these APIs are hit on PASSIVE Atlas instance in HA mode
> Following APIs do not have any issue, when hit on ACTIVE or PASSIVE instance,
> they produce the same output
> # /version
> # /status
> # /session
> # /stack
> # /patches
> # /activeSearches
> # /activeSearches/\{id}
> # /checkstate
> But the following APIs have issue, when hit on the PASSIVE instance, they
> either produce partial information, or they throw an ERROR.
> * /audits
> ** Error:
> {code:java}
> {"errorCode":"ATLAS-400-00-00A","errorMessage":"__AtlasAuditEntry:
> Unknown/invalid typename"} {code}
> * /purge
> ** Error:
> {code:java}
> {"errorCode":"ATLAS-400-00-014","errorMessage":"Type ENTITY with name
> aws_s3_v2_object does not exist"} {code}
> * /export (I understand this is by design, so can be ignored?)
> ** Error:
> {code:java}
> 2020-11-24 18:06:59,250 ERROR - [etp402695541-112:] ~ URL not supported in HA
> mode: /api/atlas/admin/export (ActiveServerFilter:120) 2020-11-24
> 18:06:59,252 INFO - [etp402695541-112:] ~ Not active. Redirecting to
> https://quasar-jfuidh-4.quasar-jfuidh.root.hwx.site:31443/api/atlas/admin/export
> (ActiveServerFilter:172) {code}
> * /import (I understand this is by design, so can be ignored?)
> ** Error:
> {code:java}
> 2020-11-24 18:07:22,702 ERROR - [etp402695541-118:] ~ URL not supported in HA
> mode: /api/atlas/admin/import (ActiveServerFilter:120) 2020-11-24
> 18:07:22,704 INFO - [etp402695541-118:] ~ Not active. Redirecting to
> https://quasar-jfuidh-4.quasar-jfuidh.root.hwx.site:31443/api/atlas/admin/import
> (ActiveServerFilter:172) {code}
> * /importfile (I understand this is by design, so can be ignored?)
> ** Error:
> {code:java}
> 2020-11-24 18:07:22,702 ERROR - [etp402695541-118:] ~ URL not supported in HA
> mode: /api/atlas/admin/import (ActiveServerFilter:120) 2020-11-24
> 18:07:22,704 INFO - [etp402695541-118:] ~ Not active. Redirecting to
> https://quasar-jfuidh-4.quasar-jfuidh.root.hwx.site:31443/api/atlas/admin/import
> (ActiveServerFilter:172) {code}
> * /audit/\{auditGuid}/details
> ** Error:
> {code:java}
> { "errorCode": "ATLAS-404-00-001", "errorMessage": "Given typename
> __AtlasAuditEntry was invalid" } {code}
> * /server/\{serverName}
> ** Error:
> {code:java}
> There was an error processing your request. It has been logged (ID
> a52953c92b609fb8). 2020-11-24 18:05:32,259 ERROR - [etp402695541-70 -
> 47fbcd4a-e547-4bba-b12b-af018e33a893:] ~ Error handling a request:
> a52953c92b609fb8 (ExceptionMapperUtil:32) java.lang.NullPointerException
> {code}
> * /metrics
> ** Error:
> {code:java}
> Returns only partial information {code}
> * /expimp/audit
> ** Error:
> {code:java}
> { "errorCode": "ATLAS-400-00-00A", "errorMessage": "__ExportImportAuditEntry:
> Unknown/invalid typename" } {code}
> The case when an /admin API is hit on the PASSIVE instance, we should either
> return a proper response, or we should redirect the request to the ACTIVE
> instance. Providing partial information/error seems to be incorrect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4043) Atlas import-kafka.sh does not import "__consumer_offsets" topic
[
https://issues.apache.org/jira/browse/ATLAS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh updated ATLAS-4043:
--
Attachment: Fixed import __consumer_offsets.png
> Atlas import-kafka.sh does not import "__consumer_offsets" topic
>
>
> Key: ATLAS-4043
> URL: https://issues.apache.org/jira/browse/ATLAS-4043
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Umesh Padashetty
>Assignee: Deep Singh
>Priority: Major
> Attachments: Fixed import __consumer_offsets.png, Screenshot
> 2020-11-20 at 11.23.47 PM.png
>
>
> Even though kafka-topics.sh --list, lists the __consumer_offsets topic as
> below, this topic alone is not imported into Atlas when import-kafka.sh
> script is run
> {code:java}
> ATLAS_ENTITIES
> ATLAS_HOOK
> ATLAS_SPARK_HOOK
> __consumer_offsets
> connect-configs
> connect-offsets
> connect-status {code}
> I did not find anything unusual in atlas/application.log. This was working
> fine before, seems like a regression.
> As per official Kafka documentation, __consumer_offsets is used to store
> information about committed offsets for each topic:partition per group of
> consumers (groupID). It is compacted topic, so data will be periodically
> compressed and only latest offsets information available.
> Since __consumer_offsets has __ in the beginning, I have created a topic
> starting with __ and ran the import-kafka script. And it works fine. So that
> leaves us with having issue with __consumer_offsets alone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-4043) Atlas import-kafka.sh does not import "__consumer_offsets" topic
[
https://issues.apache.org/jira/browse/ATLAS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240985#comment-17240985
]
Deep Singh commented on ATLAS-4043:
---
Patch Submitted for review.
https://reviews.apache.org/r/73049/
> Atlas import-kafka.sh does not import "__consumer_offsets" topic
>
>
> Key: ATLAS-4043
> URL: https://issues.apache.org/jira/browse/ATLAS-4043
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Umesh Padashetty
>Assignee: Deep Singh
>Priority: Major
> Attachments: Screenshot 2020-11-20 at 11.23.47 PM.png
>
>
> Even though kafka-topics.sh --list, lists the __consumer_offsets topic as
> below, this topic alone is not imported into Atlas when import-kafka.sh
> script is run
> {code:java}
> ATLAS_ENTITIES
> ATLAS_HOOK
> ATLAS_SPARK_HOOK
> __consumer_offsets
> connect-configs
> connect-offsets
> connect-status {code}
> I did not find anything unusual in atlas/application.log. This was working
> fine before, seems like a regression.
> As per official Kafka documentation, __consumer_offsets is used to store
> information about committed offsets for each topic:partition per group of
> consumers (groupID). It is compacted topic, so data will be periodically
> compressed and only latest offsets information available.
> Since __consumer_offsets has __ in the beginning, I have created a topic
> starting with __ and ran the import-kafka script. And it works fine. So that
> leaves us with having issue with __consumer_offsets alone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4043) Atlas import-kafka.sh does not import "__consumer_offsets" topic
[
https://issues.apache.org/jira/browse/ATLAS-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh reassigned ATLAS-4043:
-
Assignee: Deep Singh
> Atlas import-kafka.sh does not import "__consumer_offsets" topic
>
>
> Key: ATLAS-4043
> URL: https://issues.apache.org/jira/browse/ATLAS-4043
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Umesh Padashetty
>Assignee: Deep Singh
>Priority: Major
> Attachments: Screenshot 2020-11-20 at 11.23.47 PM.png
>
>
> Even though kafka-topics.sh --list, lists the __consumer_offsets topic as
> below, this topic alone is not imported into Atlas when import-kafka.sh
> script is run
> {code:java}
> ATLAS_ENTITIES
> ATLAS_HOOK
> ATLAS_SPARK_HOOK
> __consumer_offsets
> connect-configs
> connect-offsets
> connect-status {code}
> I did not find anything unusual in atlas/application.log. This was working
> fine before, seems like a regression.
> As per official Kafka documentation, __consumer_offsets is used to store
> information about committed offsets for each topic:partition per group of
> consumers (groupID). It is compacted topic, so data will be periodically
> compressed and only latest offsets information available.
> Since __consumer_offsets has __ in the beginning, I have created a topic
> starting with __ and ran the import-kafka script. And it works fine. So that
> leaves us with having issue with __consumer_offsets alone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4044) There is no way to differentiate between complete (non-shell) and incomplete (shell) entities in UI
[ https://issues.apache.org/jira/browse/ATLAS-4044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh reassigned ATLAS-4044: - Assignee: Deep Singh (was: Keval Bhatt) > There is no way to differentiate between complete (non-shell) and incomplete > (shell) entities in UI > --- > > Key: ATLAS-4044 > URL: https://issues.apache.org/jira/browse/ATLAS-4044 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Umesh Padashetty >Assignee: Deep Singh >Priority: Major > Attachments: non-shell entity.png, shell entity.png, > shell_entity_guid.png > > > Previously, we used to show the hourglass icon for the shell entities. But > this behaviour is now removed and we show the same icons for both shell and > non-shell entities. > One difference between these 2 on UI is we show only minimal information for > shell entities, like qualifiedName and typeName. But this not very obvious, > and sometimes it gets very difficult to differentiate between the two. > Attached screenshots. > It is important to note that api/atlas/v2/entity/guid/ > does return "isIncomplete": true. > So, it would be great if we can show isIncomplete parameter on the UI, which > would help the user in differentiation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4037) Empty Client ID Field in Admin audits
[
https://issues.apache.org/jira/browse/ATLAS-4037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237722#comment-17237722
]
Deep Singh commented on ATLAS-4037:
---
Review request
https://reviews.apache.org/r/73029/
Pre-commit test
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/215/console
> Empty Client ID Field in Admin audits
> -
>
> Key: ATLAS-4037
> URL: https://issues.apache.org/jira/browse/ATLAS-4037
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> TYPE_DEF_CREATE and TYPE_DEF_UPDATE have empty "*Client ID*" field.
> These should be updated for all entries that are generated by system
> Example: The following result shows where *clientID* is not available
> {code:java}
> {
> "guid": "",
> "operation": "TYPE_DEF_UPDATE",
> "params": "RELATIONSHIP,ENTITY",
> "startTime": 1605585786541,
> "endTime": 1605585786747,
> "result": "<>",
> "resultCount": 3
> }
> {code}
>
> Result where all fiends are available:
> {code:java}
> {
> "guid": "",
> "userName": "",
> "operation": "TYPE_DEF_UPDATE",
> "params": "CLASSIFICATION",
> "startTime": 1605700613879,
> "endTime": 1605700614297,
> "clientId": "",
> "result": "",
> "resultCount": 1
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-4036) Empty "Users" Field in Admin audits
[
https://issues.apache.org/jira/browse/ATLAS-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237721#comment-17237721
]
Deep Singh commented on ATLAS-4036:
---
Review request
https://reviews.apache.org/r/73029/
Pre-commit test
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/215/console
> Empty "Users" Field in Admin audits
> ---
>
> Key: ATLAS-4036
> URL: https://issues.apache.org/jira/browse/ATLAS-4036
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core, atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> TYPE_DEF_CREATE, TYPE_DEF_UPDATE, SERVER_START and SERVER_STATE_ACTIVE have
> empty "*Users*" field.
> If the changes are being done by the system/admin the same should be updated
> there like "System generated"/ "System"/"Admin" or something similar
> Example: Here is a *SERVER_START* operation audit. Here There is no
> *userName* field, resulting in empty Users in UI
> {code}
> [
> {
> "guid": "",
> "operation": "SERVER_START",
> "startTime": 1605585671419,
> "endTime": 1605585837730,
> "clientId": "",
> "resultCount": 0
> }
> ]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4036) Empty "Users" Field in Admin audits
[
https://issues.apache.org/jira/browse/ATLAS-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh reassigned ATLAS-4036:
-
Assignee: Deep Singh (was: Mandar Ambawane)
> Empty "Users" Field in Admin audits
> ---
>
> Key: ATLAS-4036
> URL: https://issues.apache.org/jira/browse/ATLAS-4036
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core, atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> TYPE_DEF_CREATE, TYPE_DEF_UPDATE, SERVER_START and SERVER_STATE_ACTIVE have
> empty "*Users*" field.
> If the changes are being done by the system/admin the same should be updated
> there like "System generated"/ "System"/"Admin" or something similar
> Example: Here is a *SERVER_START* operation audit. Here There is no
> *userName* field, resulting in empty Users in UI
> {code}
> [
> {
> "guid": "",
> "operation": "SERVER_START",
> "startTime": 1605585671419,
> "endTime": 1605585837730,
> "clientId": "",
> "resultCount": 0
> }
> ]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ATLAS-4035) End-time in SERVER_STATE_ACTIVE audits might be N/A instead of the value since it has same value as start time
[
https://issues.apache.org/jira/browse/ATLAS-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-4035.
---
Resolution: Not A Bug
> End-time in SERVER_STATE_ACTIVE audits might be N/A instead of the value
> since it has same value as start time
> --
>
> Key: ATLAS-4035
> URL: https://issues.apache.org/jira/browse/ATLAS-4035
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core, atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> The Start Time and End Time in the {color:#172b4d}SERVER_STATE_ACTIVE is
> always same. Its better to remove the End time column or update it as
> N/A{color}
> Response of SERVER_STATE_ACTIVE audit
> {code:java}
> {
> "guid": "",
> "operation": "SERVER_STATE_ACTIVE",
> "startTime": 1604902481169,
> "endTime": 1604902481169,
> "clientId": "",
> "resultCount": 0
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-4035) End-time in SERVER_STATE_ACTIVE audits might be N/A instead of the value since it has same value as start time
[
https://issues.apache.org/jira/browse/ATLAS-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235919#comment-17235919
]
Deep Singh commented on ATLAS-4035:
---
SERVER_STATE_ACTIVE is an event(not process) and we are trying to list it in a
table designed for the listing of processes. For end-user everything on the
list is a process, with a defined start time and end time.
An event can be considered as a special process which consumes no time and has
the same start time and end time.
If we have a different representation for the event(special process) it will
confuse the user.
Closing this bug. Works as designed.
> End-time in SERVER_STATE_ACTIVE audits might be N/A instead of the value
> since it has same value as start time
> --
>
> Key: ATLAS-4035
> URL: https://issues.apache.org/jira/browse/ATLAS-4035
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core, atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Deep Singh
>Priority: Major
>
> The Start Time and End Time in the {color:#172b4d}SERVER_STATE_ACTIVE is
> always same. Its better to remove the End time column or update it as
> N/A{color}
> Response of SERVER_STATE_ACTIVE audit
> {code:java}
> {
> "guid": "",
> "operation": "SERVER_STATE_ACTIVE",
> "startTime": 1604902481169,
> "endTime": 1604902481169,
> "clientId": "",
> "resultCount": 0
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4019) Audit improvement to store only delta
Deep Singh created ATLAS-4019: - Summary: Audit improvement to store only delta Key: ATLAS-4019 URL: https://issues.apache.org/jira/browse/ATLAS-4019 Project: Atlas Issue Type: Improvement Components: atlas-intg Reporter: Deep Singh Assignee: Deep Singh For any entity operation after the entity is created - e.g. add a column, update an attribute. - the entire updated entity is saved in Hbase audits (for each operation). Only the delta between two entity states needs to be stored. Update routine could be called for an individual entity or for a list of entities. Update-request may contain a full entity object or partial object(with only updated attributes or relations). There could be updates in entity core attributes, relationship attributes, custom attributes, or associated classifications. In all the cases today we store the entire updated entity in Hbase audits. Improvement is required so that only the updated information is stored in Hbase audits instead of the entire entity object. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3987) Atlas client export API does not pass server response code in case of failure
[ https://issues.apache.org/jira/browse/ATLAS-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-3987. --- Resolution: Fixed > Atlas client export API does not pass server response code in case of failure > - > > Key: ATLAS-3987 > URL: https://issues.apache.org/jira/browse/ATLAS-3987 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > > For Atlas export, Admin priveledges are required. If such priveledges are not > there, then Export API fails with 403 status. However, the atlasClient does > not pass this information in the exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3987) Atlas client export API does not pass server response code in case of failure
Deep Singh created ATLAS-3987: - Summary: Atlas client export API does not pass server response code in case of failure Key: ATLAS-3987 URL: https://issues.apache.org/jira/browse/ATLAS-3987 Project: Atlas Issue Type: Bug Components: atlas-intg Reporter: Deep Singh Assignee: Deep Singh For Atlas export, Admin priveledges are required. If such priveledges are not there, then Export API fails with 403 status. However, the atlasClient does not pass this information in the exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3964) Atlas UI displays large numbers incorrectly
[ https://issues.apache.org/jira/browse/ATLAS-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-3964. --- Resolution: Fixed > Atlas UI displays large numbers incorrectly > --- > > Key: ATLAS-3964 > URL: https://issues.apache.org/jira/browse/ATLAS-3964 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Attachments: details.doc > > > In the case of a type that has an attribute of type Long, and it contains a > very large number, the Atlas UI does not show the correct number. > Steps to reproduce: > From the command line: > # create an entity type which has a field of type long > # create an entity of this same type and the value of the field should be > "1085741226505763426" > # Atlas will respond with the created entity, you will see that the field > value is correct > # now open the Atlas website and search for this same entity; you will see > the value displayed is "1085741226505763500" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3964) Atlas UI displays large numbers incorrectly
[ https://issues.apache.org/jira/browse/ATLAS-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208454#comment-17208454 ] Deep Singh commented on ATLAS-3964: --- Review request raised. https://reviews.apache.org/r/72932 > Atlas UI displays large numbers incorrectly > --- > > Key: ATLAS-3964 > URL: https://issues.apache.org/jira/browse/ATLAS-3964 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Attachments: details.doc > > > In the case of a type that has an attribute of type Long, and it contains a > very large number, the Atlas UI does not show the correct number. > Steps to reproduce: > From the command line: > # create an entity type which has a field of type long > # create an entity of this same type and the value of the field should be > "1085741226505763426" > # Atlas will respond with the created entity, you will see that the field > value is correct > # now open the Atlas website and search for this same entity; you will see > the value displayed is "1085741226505763500" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (ATLAS-3964) Atlas UI displays large numbers incorrectly
[ https://issues.apache.org/jira/browse/ATLAS-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh reopened ATLAS-3964: --- Regression was observed in counts of entities. That was because all sorts of numbers were parsed into string. Only those numbers where information is being lost must be passed into a string. > Atlas UI displays large numbers incorrectly > --- > > Key: ATLAS-3964 > URL: https://issues.apache.org/jira/browse/ATLAS-3964 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Attachments: details.doc > > > In the case of a type that has an attribute of type Long, and it contains a > very large number, the Atlas UI does not show the correct number. > Steps to reproduce: > From the command line: > # create an entity type which has a field of type long > # create an entity of this same type and the value of the field should be > "1085741226505763426" > # Atlas will respond with the created entity, you will see that the field > value is correct > # now open the Atlas website and search for this same entity; you will see > the value displayed is "1085741226505763500" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3965) Relationships with null values are missing in entity relationshipAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh resolved ATLAS-3965.
---
Resolution: Fixed
> Relationships with null values are missing in entity relationshipAttributes
> ---
>
> Key: ATLAS-3965
> URL: https://issues.apache.org/jira/browse/ATLAS-3965
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
> Attachments:
> 0002-ATLAS-3965-Relationships-with-null-values-are-missin.patch
>
>
> kafka_topic typedef has relationship attributes:
> {code:java}
> inputToProcesses
> pipeline
> schema
> model
> avroSchema
> meanings
> outputFromProcesses{code}
> When a kafka topic entity is created using import-kafka.sh script, the entity
> doesn't have all relationship attributes :
> {code:java}
> "relationshipAttributes": {
> "inputToProcesses": [ ],
> "schema": [ ],
> "avroSchema": [ ],
> "meanings": [ ],
> "outputFromProcesses": [ ]}, {code}
> The missing ones are "pipeline" and "model" .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-3966) Newer value for a property in application configuration is not loaded
[ https://issues.apache.org/jira/browse/ATLAS-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205678#comment-17205678 ] Deep Singh commented on ATLAS-3966: --- Patch Attached PreCpmmit Build Test passed https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/46// Review board link https://reviews.apache.org/r/72921/ > Newer value for a property in application configuration is not loaded > -- > > Key: ATLAS-3966 > URL: https://issues.apache.org/jira/browse/ATLAS-3966 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Attachments: > 0003-ATLAS-3966-Newer-value-for-atlas.metadata.namespace-.patch > > > If a property is defined multiple times with different values in the > application configuration file, while loading the proper, the oldest value > is read. Technically this is correct behavior. However many times, instead of > updating the value of a particular property, its convenient to just append > the property again with a new value in the configuration file. > This ask is specifically for a property called "atlas.metadata.namespace" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3966) Newer value for a property in application configuration is not loaded
[ https://issues.apache.org/jira/browse/ATLAS-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh updated ATLAS-3966: -- Attachment: 0003-ATLAS-3966-Newer-value-for-atlas.metadata.namespace-.patch > Newer value for a property in application configuration is not loaded > -- > > Key: ATLAS-3966 > URL: https://issues.apache.org/jira/browse/ATLAS-3966 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Attachments: > 0003-ATLAS-3966-Newer-value-for-atlas.metadata.namespace-.patch > > > If a property is defined multiple times with different values in the > application configuration file, while loading the proper, the oldest value > is read. Technically this is correct behavior. However many times, instead of > updating the value of a particular property, its convenient to just append > the property again with a new value in the configuration file. > This ask is specifically for a property called "atlas.metadata.namespace" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3966) Newer value for a property in application configuration is not loaded
Deep Singh created ATLAS-3966: - Summary: Newer value for a property in application configuration is not loaded Key: ATLAS-3966 URL: https://issues.apache.org/jira/browse/ATLAS-3966 Project: Atlas Issue Type: Bug Components: atlas-intg Reporter: Deep Singh Assignee: Deep Singh If a property is defined multiple times with different values in the application configuration file, while loading the proper, the oldest value is read. Technically this is correct behavior. However many times, instead of updating the value of a particular property, its convenient to just append the property again with a new value in the configuration file. This ask is specifically for a property called "atlas.metadata.namespace" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3965) Relationships with null values are missing in entity relationshipAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205086#comment-17205086
]
Deep Singh commented on ATLAS-3965:
---
Patch Attached
PreCpmmit Build Test passed
[https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/44//]
> Relationships with null values are missing in entity relationshipAttributes
> ---
>
> Key: ATLAS-3965
> URL: https://issues.apache.org/jira/browse/ATLAS-3965
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
> Attachments:
> 0002-ATLAS-3965-Relationships-with-null-values-are-missin.patch
>
>
> kafka_topic typedef has relationship attributes:
> {code:java}
> inputToProcesses
> pipeline
> schema
> model
> avroSchema
> meanings
> outputFromProcesses{code}
> When a kafka topic entity is created using import-kafka.sh script, the entity
> doesn't have all relationship attributes :
> {code:java}
> "relationshipAttributes": {
> "inputToProcesses": [ ],
> "schema": [ ],
> "avroSchema": [ ],
> "meanings": [ ],
> "outputFromProcesses": [ ]}, {code}
> The missing ones are "pipeline" and "model" .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3965) Relationships with null values are missing in entity relationshipAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh updated ATLAS-3965:
--
Attachment: 0002-ATLAS-3965-Relationships-with-null-values-are-missin.patch
> Relationships with null values are missing in entity relationshipAttributes
> ---
>
> Key: ATLAS-3965
> URL: https://issues.apache.org/jira/browse/ATLAS-3965
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
> Attachments:
> 0002-ATLAS-3965-Relationships-with-null-values-are-missin.patch
>
>
> kafka_topic typedef has relationship attributes:
> {code:java}
> inputToProcesses
> pipeline
> schema
> model
> avroSchema
> meanings
> outputFromProcesses{code}
> When a kafka topic entity is created using import-kafka.sh script, the entity
> doesn't have all relationship attributes :
> {code:java}
> "relationshipAttributes": {
> "inputToProcesses": [ ],
> "schema": [ ],
> "avroSchema": [ ],
> "meanings": [ ],
> "outputFromProcesses": [ ]}, {code}
> The missing ones are "pipeline" and "model" .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3965) Relationships with null values are missing in entity relationshipAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh updated ATLAS-3965:
--
Description:
kafka_topic typedef has relationship attributes:
{code:java}
inputToProcesses
pipeline
schema
model
avroSchema
meanings
outputFromProcesses{code}
When a kafka topic entity is created using import-kafka.sh script, the entity
doesn't have all relationship attributes :
{code:java}
"relationshipAttributes": {
"inputToProcesses": [ ],
"schema": [ ],
"avroSchema": [ ],
"meanings": [ ],
"outputFromProcesses": [ ]}, {code}
The missing ones are "pipeline" and "model" .
was:
kafka_topic typedef has relationship attributes:
{code:java}inputToProcesses
pipeline
schema
model
avroSchema
meanings
outputFromProcesses\{code}
When a kafka topic entity is created using import-kafka.sh script, the entity
doesn't have all relationship attributes :
{code:java}"relationshipAttributes":
{ "inputToProcesses": [ ], "schema": [ ], "avroSchema": [ ], "meanings": [ ],
"outputFromProcesses": [ ]}
, \{code}
The missing ones are "pipeline" and "model" .
Among the null valued relationship Attributes , "pipeline" and "model" are of
cardinality "SINGLE"
> Relationships with null values are missing in entity relationshipAttributes
> ---
>
> Key: ATLAS-3965
> URL: https://issues.apache.org/jira/browse/ATLAS-3965
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
>
> kafka_topic typedef has relationship attributes:
> {code:java}
> inputToProcesses
> pipeline
> schema
> model
> avroSchema
> meanings
> outputFromProcesses{code}
> When a kafka topic entity is created using import-kafka.sh script, the entity
> doesn't have all relationship attributes :
> {code:java}
> "relationshipAttributes": {
> "inputToProcesses": [ ],
> "schema": [ ],
> "avroSchema": [ ],
> "meanings": [ ],
> "outputFromProcesses": [ ]}, {code}
> The missing ones are "pipeline" and "model" .
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3965) Relationships with null values are missing in entity relationshipAttributes
[
https://issues.apache.org/jira/browse/ATLAS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deep Singh updated ATLAS-3965:
--
Description:
kafka_topic typedef has relationship attributes:
{code:java}inputToProcesses
pipeline
schema
model
avroSchema
meanings
outputFromProcesses\{code}
When a kafka topic entity is created using import-kafka.sh script, the entity
doesn't have all relationship attributes :
{code:java}"relationshipAttributes":
{ "inputToProcesses": [ ], "schema": [ ], "avroSchema": [ ], "meanings": [ ],
"outputFromProcesses": [ ]}
, \{code}
The missing ones are "pipeline" and "model" .
Among the null valued relationship Attributes , "pipeline" and "model" are of
cardinality "SINGLE"
was:
kafka_topic typedef has relationship attributes:
{code:java}
inputToProcesses
pipeline
schema
model
avroSchema
meanings
outputFromProcesses\{code}
When a kafka topic entity is created using import-kafka.sh script, the entity
doesn't have all relationship attributes :
{code:java}
"relationshipAttributes": {
"inputToProcesses": [ ],
"schema": [ ],
"avroSchema": [ ],
"meanings": [ ],
"outputFromProcesses": [ ]}, \{code}
The missing ones are "pipeline" and "model" .
Among the null valued relationship Attributes , "pipeline" and "model" are of
cardinality "SINGLE"
> Relationships with null values are missing in entity relationshipAttributes
> ---
>
> Key: ATLAS-3965
> URL: https://issues.apache.org/jira/browse/ATLAS-3965
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Reporter: Deep Singh
>Assignee: Deep Singh
>Priority: Major
>
> kafka_topic typedef has relationship attributes:
> {code:java}inputToProcesses
> pipeline
> schema
> model
> avroSchema
> meanings
> outputFromProcesses\{code}
>
> When a kafka topic entity is created using import-kafka.sh script, the entity
> doesn't have all relationship attributes :
>
> {code:java}"relationshipAttributes":
> { "inputToProcesses": [ ], "schema": [ ], "avroSchema": [ ], "meanings": [ ],
> "outputFromProcesses": [ ]}
> , \{code}
>
> The missing ones are "pipeline" and "model" .
> Among the null valued relationship Attributes , "pipeline" and "model" are of
> cardinality "SINGLE"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-3965) Relationships with null values are missing in entity relationshipAttributes
Deep Singh created ATLAS-3965:
-
Summary: Relationships with null values are missing in entity
relationshipAttributes
Key: ATLAS-3965
URL: https://issues.apache.org/jira/browse/ATLAS-3965
Project: Atlas
Issue Type: Bug
Components: atlas-intg
Reporter: Deep Singh
Assignee: Deep Singh
kafka_topic typedef has relationship attributes:
{code:java}
inputToProcesses
pipeline
schema
model
avroSchema
meanings
outputFromProcesses\{code}
When a kafka topic entity is created using import-kafka.sh script, the entity
doesn't have all relationship attributes :
{code:java}
"relationshipAttributes": {
"inputToProcesses": [ ],
"schema": [ ],
"avroSchema": [ ],
"meanings": [ ],
"outputFromProcesses": [ ]}, \{code}
The missing ones are "pipeline" and "model" .
Among the null valued relationship Attributes , "pipeline" and "model" are of
cardinality "SINGLE"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-3964) Atlas UI displays large numbers incorrectly
[ https://issues.apache.org/jira/browse/ATLAS-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh reassigned ATLAS-3964: - Assignee: Deep Singh > Atlas UI displays large numbers incorrectly > --- > > Key: ATLAS-3964 > URL: https://issues.apache.org/jira/browse/ATLAS-3964 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Attachments: details.doc > > > In the case of a type that has an attribute of type Long, and it contains a > very large number, the Atlas UI does not show the correct number. > Steps to reproduce: > From the command line: > # create an entity type which has a field of type long > # create an entity of this same type and the value of the field should be > "1085741226505763426" > # Atlas will respond with the created entity, you will see that the field > value is correct > # now open the Atlas website and search for this same entity; you will see > the value displayed is "1085741226505763500" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3964) Atlas UI displays large numbers incorrectly
[ https://issues.apache.org/jira/browse/ATLAS-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17204555#comment-17204555 ] Deep Singh commented on ATLAS-3964: --- The problem is with client-side parsing. It's not an issue with our UI code, its a problem on browser's javascript engine. Javascript cannot handle values bigger than Number.MAX_SAFE_INTEGER (2^53 == 9 007 199 254 740 992). long is 2^64, so for any number greater than 2^53, javascript does internal optimization to maintain sanity. > Atlas UI displays large numbers incorrectly > --- > > Key: ATLAS-3964 > URL: https://issues.apache.org/jira/browse/ATLAS-3964 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Reporter: Deep Singh >Priority: Major > Attachments: details.doc > > > In the case of a type that has an attribute of type Long, and it contains a > very large number, the Atlas UI does not show the correct number. > Steps to reproduce: > From the command line: > # create an entity type which has a field of type long > # create an entity of this same type and the value of the field should be > "1085741226505763426" > # Atlas will respond with the created entity, you will see that the field > value is correct > # now open the Atlas website and search for this same entity; you will see > the value displayed is "1085741226505763500" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3964) Atlas UI displays large numbers incorrectly
Deep Singh created ATLAS-3964: - Summary: Atlas UI displays large numbers incorrectly Key: ATLAS-3964 URL: https://issues.apache.org/jira/browse/ATLAS-3964 Project: Atlas Issue Type: Bug Components: atlas-webui Reporter: Deep Singh Attachments: details.doc In the case of a type that has an attribute of type Long, and it contains a very large number, the Atlas UI does not show the correct number. Steps to reproduce: >From the command line: # create an entity type which has a field of type long # create an entity of this same type and the value of the field should be "1085741226505763426" # Atlas will respond with the created entity, you will see that the field value is correct # now open the Atlas website and search for this same entity; you will see the value displayed is "1085741226505763500" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3944) Logs of import scripts should go to Atlas default log directory
[ https://issues.apache.org/jira/browse/ATLAS-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-3944. --- Resolution: Fixed > Logs of import scripts should go to Atlas default log directory > --- > > Key: ATLAS-3944 > URL: https://issues.apache.org/jira/browse/ATLAS-3944 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Affects Versions: trunk, 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Fix For: trunk > > Attachments: 0001-Setting-default-log-directory-to-var-log-atlas.patch > > > Import scripts like import-hbase.sh try to dump logs in a log directory > inside the base directory. Most of the users running import scripts have > read-only access to the base directory, this leads to the following error > exception. > java.io.FileNotFoundException: > /opt/cloudera/parcels/CDH/lib/atlas/logs/import-hbase.log (No such file or > directory) > Although there is a mechanism to set the log location to the desired > directory using env variable ATLAS_LOG_DIR, but by default, the script must > try to dump logs in Atlas default logs directory (/var/log/atlas) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3944) Logs of import scripts should go to Atlas default log directory
[ https://issues.apache.org/jira/browse/ATLAS-3944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh updated ATLAS-3944: -- Attachment: 0001-Setting-default-log-directory-to-var-log-atlas.patch > Logs of import scripts should go to Atlas default log directory > --- > > Key: ATLAS-3944 > URL: https://issues.apache.org/jira/browse/ATLAS-3944 > Project: Atlas > Issue Type: Bug > Components: atlas-intg >Affects Versions: trunk, 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Fix For: trunk > > Attachments: 0001-Setting-default-log-directory-to-var-log-atlas.patch > > > Import scripts like import-hbase.sh try to dump logs in a log directory > inside the base directory. Most of the users running import scripts have > read-only access to the base directory, this leads to the following error > exception. > java.io.FileNotFoundException: > /opt/cloudera/parcels/CDH/lib/atlas/logs/import-hbase.log (No such file or > directory) > Although there is a mechanism to set the log location to the desired > directory using env variable ATLAS_LOG_DIR, but by default, the script must > try to dump logs in Atlas default logs directory (/var/log/atlas) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3944) Logs of import scripts should go to Atlas default log directory
Deep Singh created ATLAS-3944: - Summary: Logs of import scripts should go to Atlas default log directory Key: ATLAS-3944 URL: https://issues.apache.org/jira/browse/ATLAS-3944 Project: Atlas Issue Type: Bug Components: atlas-intg Affects Versions: 2.1.0, trunk Reporter: Deep Singh Assignee: Deep Singh Fix For: trunk Import scripts like import-hbase.sh try to dump logs in a log directory inside the base directory. Most of the users running import scripts have read-only access to the base directory, this leads to the following error exception. java.io.FileNotFoundException: /opt/cloudera/parcels/CDH/lib/atlas/logs/import-hbase.log (No such file or directory) Although there is a mechanism to set the log location to the desired directory using env variable ATLAS_LOG_DIR, but by default, the script must try to dump logs in Atlas default logs directory (/var/log/atlas) -- This message was sent by Atlassian Jira (v8.3.4#803005)
