Re: [DRAFT] Board report for Apache Atlas: June 2019
Thanks Madhan for drafting the report. +1 Thanks, Sarath On Mon, Jun 17, 2019 at 10:49 AM Madhan Neethiraj wrote: > Atlas team, > > Please review the draft board report below and send your feedback/comments. > > Thanks, > Madhan > > > ## Description: > Apache Atlas is a scalable and extensible set of core foundational > governance services that enables enterprises to effectively and > efficiently > meet their compliance requirements within Hadoop and allows integration > with > the complete enterprise data ecosystem > > ## Issues: > There are no issues requiring board attention at this time. > > ## Activity: > - Released 2.0.0 major version, to support Hadoop 3, HBase 2, Solr 7, > Kafka 2, Hive 3 > - Released 1.2.0 maintenance version, with 170 fixes/enhancements > - added Atlas hook for Hive Meta Store (HMS) > - added Atlas hook for Apache Impala > - added framework to track status of typedef and Java patches > - added a REST endpoint and UI to publish runtime stats from Atlas server > - UI improvements in rendering relationships > - updated QuickStart tutorial to replace use of object-ref attributes > with relationships > - updated RDBMS types to replace use of object-ref attributes with > relationships, for better performance > - added type-definitions to capture metadata and lineage from Spark > - enhanced Hive hook to capture DDL statements > - working on quick-search functionality, to provide a simpler search > interface > - working on 0.8.4 maintenance release, likely to be the last release > for 0.8 line > > ## Health report: > - 12 new contributors added in last 3 months: > Carol Drummond, Chaitali Borol, Diego Marino Monetti, Kirankumar D G, > Li Na, Mandar Ambawan, Mayank Jain, Merryle Wang, > Rahul Kurup, Umesh S. Padashetty, Xinran Yu, Yu-Hsin Shih > > ## PMC changes: > - Currently 34 PMC members > - 1 new PMC member added in last 3 months, on 5/13/2019 > - Last addition to PMC role was Ashutosh Mestry > > > ## Committer base changes: > - Currently 38 committers > - No new committers added in last 3 months > - Last addition to committer role was on 10/15/2018 > > ## Releases: > 0.8.4plan to release by 06/31/2019 > 1.2.0was released on 06/12/2019 > 2.0.0was released on 05/14/2019 > 0.8.3was released on 10/31/2018 > 1.1.0was released on 09/17/2018 > 1.0.0was released on 06/02/2018 > 0.8.2was released on 02/05/2018 > 1.0.0-alpha was released on 01/25/2018 > 0.8.1was released on 08/29/2017 > 0.8-incubating was released on 03/16/2017 > 0.7.1-incubating was released on 01/26/2017 > 0.7-incubating was released on 07/09/2016 > 0.6-incubating was released on 12/31/2015 > 0.5-incubating was released on 07/11/2015 > > > >
Re: [VOTE] Release Apache Atlas version 0.8.4 - rc1
+1 for Atlas 0.8.4 rc1. - Validated checksum and signature - built atlas with embedded-hbase-solr profile - ran Quickstart and validated that basic functionalities work. Thanks, Sarath On Mon, Jun 17, 2019 at 8:06 PM Madhan Neethiraj wrote: > +1 for Apache Atlas 0.8.4 release candidate #1. > > Performed the following: > - downloaded apache-atlas-0.8.4-sources.tar.gz and verified the signature > - built and installed Atlas for embedded HBase, Solr profile > - started Atlas with embedded HBase and Solr > - ran QuickStart successfully > - performed number of searches - both basic and DSL > - added/updated/deleted few 'Favorite Searches' - both basic and DSL > - created/deleted tags, associated/disassociated tags with entities > > Thanks, > Madhan > > > On 6/13/19, 4:09 AM, "Keval Bhatt" wrote: > > Atlas team, > > Apache Atlas 0.8.4 release candidate #1 is now available for a vote > within > dev community. > > Following commits went into branch-0.8 since the last release candidate > (rc0): > > ATLAS-3231: Lineage Icon updated. > > Links to the release artifacts are given below. Please review and vote. > > The vote will be open for at least 72 hours or until necessary votes > are > reached. > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and reason why) > > > Thanks, > Keval Bhatt > > > List of issues addressed in this release: > > https://issues.apache.org/jira/issues/?jql=project%3DATLAS%20AND%20resolution%3DFixed%20AND%20fixVersion%3D0.8.4%20ORDER%20BY%20key%20DESC > > Git tag for the release: > https://github.com/apache/atlas/tree/release-0.8.4-rc1 > > Sources for the release: > > https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz > > Source release verification: > PGP Signature: > > https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz.asc > > MD5 Hash: > > https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz.md5 > > SHA512 Hash: > > https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz.sha512 > > > Keys to verify the signature of the release artifacts are available at: > https://dist.apache.org/repos/dist/dev/atlas/KEYS > > > >
[jira] [Updated] (ATLAS-3249) UI: "Show more"/"Show less" button for columns with multiple entries within a cell
[ https://issues.apache.org/jira/browse/ATLAS-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Binit Gutka updated ATLAS-3249: --- Attachment: (was: ATLAS-3249_1.patch) > UI: "Show more"/"Show less" button for columns with multiple entries within a > cell > -- > > Key: ATLAS-3249 > URL: https://issues.apache.org/jira/browse/ATLAS-3249 > Project: Atlas > Issue Type: Improvement >Reporter: Rahul Kurup >Assignee: Binit Gutka >Priority: Minor > Attachments: ATLAS-3249_2.patch, showmoreshowless.png > > > There should be a "Show more/Show less" button in a scenario where there are > a lot of entries within a cell. > For example, in the attached screenshot below, if the user searches for a > hive_table and selects "Columns" as one of the displayed parameters, you can > see that one search result may have multiple entries within the "Columns" > column that increases the width of the search result and disrupts the UI > alignment. > A Show More/Show Less button that expands or compresses the displayed entries > of the "Columns" table similar to the Classification would fix this alignment > issue. !showmoreshowless.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3249) UI: "Show more"/"Show less" button for columns with multiple entries within a cell
[ https://issues.apache.org/jira/browse/ATLAS-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Binit Gutka updated ATLAS-3249: --- Attachment: ATLAS-3249_2.patch > UI: "Show more"/"Show less" button for columns with multiple entries within a > cell > -- > > Key: ATLAS-3249 > URL: https://issues.apache.org/jira/browse/ATLAS-3249 > Project: Atlas > Issue Type: Improvement >Reporter: Rahul Kurup >Assignee: Binit Gutka >Priority: Minor > Attachments: ATLAS-3249_1.patch, ATLAS-3249_2.patch, > showmoreshowless.png > > > There should be a "Show more/Show less" button in a scenario where there are > a lot of entries within a cell. > For example, in the attached screenshot below, if the user searches for a > hive_table and selects "Columns" as one of the displayed parameters, you can > see that one search result may have multiple entries within the "Columns" > column that increases the width of the search result and disrupts the UI > alignment. > A Show More/Show Less button that expands or compresses the displayed entries > of the "Columns" table similar to the Classification would fix this alignment > issue. !showmoreshowless.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [DRAFT] Board report for Apache Atlas: June 2019
Thanks Madhan for putting this together. Looks good to me. +1 for the draft Regards Nixon On Mon, Jun 17, 2019, 11:19 PM Madhan Neethiraj wrote: > Atlas team, > > Please review the draft board report below and send your feedback/comments. > > Thanks, > Madhan > > > ## Description: > Apache Atlas is a scalable and extensible set of core foundational > governance services that enables enterprises to effectively and > efficiently > meet their compliance requirements within Hadoop and allows integration > with > the complete enterprise data ecosystem > > ## Issues: > There are no issues requiring board attention at this time. > > ## Activity: > - Released 2.0.0 major version, to support Hadoop 3, HBase 2, Solr 7, > Kafka 2, Hive 3 > - Released 1.2.0 maintenance version, with 170 fixes/enhancements > - added Atlas hook for Hive Meta Store (HMS) > - added Atlas hook for Apache Impala > - added framework to track status of typedef and Java patches > - added a REST endpoint and UI to publish runtime stats from Atlas server > - UI improvements in rendering relationships > - updated QuickStart tutorial to replace use of object-ref attributes > with relationships > - updated RDBMS types to replace use of object-ref attributes with > relationships, for better performance > - added type-definitions to capture metadata and lineage from Spark > - enhanced Hive hook to capture DDL statements > - working on quick-search functionality, to provide a simpler search > interface > - working on 0.8.4 maintenance release, likely to be the last release > for 0.8 line > > ## Health report: > - 12 new contributors added in last 3 months: > Carol Drummond, Chaitali Borol, Diego Marino Monetti, Kirankumar D G, > Li Na, Mandar Ambawan, Mayank Jain, Merryle Wang, > Rahul Kurup, Umesh S. Padashetty, Xinran Yu, Yu-Hsin Shih > > ## PMC changes: > - Currently 34 PMC members > - 1 new PMC member added in last 3 months, on 5/13/2019 > - Last addition to PMC role was Ashutosh Mestry > > > ## Committer base changes: > - Currently 38 committers > - No new committers added in last 3 months > - Last addition to committer role was on 10/15/2018 > > ## Releases: > 0.8.4plan to release by 06/31/2019 > 1.2.0was released on 06/12/2019 > 2.0.0was released on 05/14/2019 > 0.8.3was released on 10/31/2018 > 1.1.0was released on 09/17/2018 > 1.0.0was released on 06/02/2018 > 0.8.2was released on 02/05/2018 > 1.0.0-alpha was released on 01/25/2018 > 0.8.1was released on 08/29/2017 > 0.8-incubating was released on 03/16/2017 > 0.7.1-incubating was released on 01/26/2017 > 0.7-incubating was released on 07/09/2016 > 0.6-incubating was released on 12/31/2015 > 0.5-incubating was released on 07/11/2015 > > > >
Re: [VOTE] Release Apache Atlas version 0.8.4 - rc1
+1 for Apache Atlas 0.8.4 release candidate #1. Performed the following: - downloaded apache-atlas-0.8.4-sources.tar.gz and verified the signature - built and installed Atlas for embedded HBase, Solr profile - started Atlas with embedded HBase and Solr - ran QuickStart successfully - performed number of searches - both basic and DSL - added/updated/deleted few 'Favorite Searches' - both basic and DSL - created/deleted tags, associated/disassociated tags with entities Thanks, Madhan On 6/13/19, 4:09 AM, "Keval Bhatt" wrote: Atlas team, Apache Atlas 0.8.4 release candidate #1 is now available for a vote within dev community. Following commits went into branch-0.8 since the last release candidate (rc0): ATLAS-3231: Lineage Icon updated. Links to the release artifacts are given below. Please review and vote. The vote will be open for at least 72 hours or until necessary votes are reached. [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, Keval Bhatt List of issues addressed in this release: https://issues.apache.org/jira/issues/?jql=project%3DATLAS%20AND%20resolution%3DFixed%20AND%20fixVersion%3D0.8.4%20ORDER%20BY%20key%20DESC Git tag for the release: https://github.com/apache/atlas/tree/release-0.8.4-rc1 Sources for the release: https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz Source release verification: PGP Signature: https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz.asc MD5 Hash: https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz.md5 SHA512 Hash: https://dist.apache.org/repos/dist/dev/atlas/0.8.4-rc1/apache-atlas-0.8.4-sources.tar.gz.sha512 Keys to verify the signature of the release artifacts are available at: https://dist.apache.org/repos/dist/dev/atlas/KEYS
[GitHub] [atlas] fusonghe commented on issue #39: ATLAS-3071:Add Functionalities to Collect Notification Metrics
fusonghe commented on issue #39: ATLAS-3071:Add Functionalities to Collect Notification Metrics URL: https://github.com/apache/atlas/pull/39#issuecomment-502916115 hello Hive deletes the table but still shows the existence of this table in altas @sarathsubramanian @mahaha000 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (ATLAS-3289) Add a configuration to populate dynamic attributes. If the config is disabled, read the value from the Hook
[ https://issues.apache.org/jira/browse/ATLAS-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang updated ATLAS-3289: Remaining Estimate: 72h Original Estimate: 72h Summary: Add a configuration to populate dynamic attributes. If the config is disabled, read the value from the Hook (was: Add a configuration to population dynamic attributes. If the config is disabled, read the value from the Hook) > Add a configuration to populate dynamic attributes. If the config is > disabled, read the value from the Hook > --- > > Key: ATLAS-3289 > URL: https://issues.apache.org/jira/browse/ATLAS-3289 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 72h > Remaining Estimate: 72h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3289) Add a configuration to population dynamic attributes. If the config is disabled, read the value from the Hook
Aadarsh Jajodia created ATLAS-3289: -- Summary: Add a configuration to population dynamic attributes. If the config is disabled, read the value from the Hook Key: ATLAS-3289 URL: https://issues.apache.org/jira/browse/ATLAS-3289 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-3288) Implement the DynamicAttributesEvaluator method
[ https://issues.apache.org/jira/browse/ATLAS-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang reassigned ATLAS-3288: --- Assignee: Merryle Wang > Implement the DynamicAttributesEvaluator method > > > Key: ATLAS-3288 > URL: https://issues.apache.org/jira/browse/ATLAS-3288 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 672h > Remaining Estimate: 672h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
[
https://issues.apache.org/jira/browse/ATLAS-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Merryle Wang updated ATLAS-3285:
Remaining Estimate: 168h
Original Estimate: 168h
> Defining the template version for QualifiedName in the models file for hive
> entity types
>
>
> Key: ATLAS-3285
> URL: https://issues.apache.org/jira/browse/ATLAS-3285
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Aadarsh Jajodia
>Assignee: Merryle Wang
>Priority: Major
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> An example of a Dynamic Attribute for a Hive Column looks something like this
> in the hive models file
> {code:java}
> "options": {
> "dynAttribute:qualifiedName":
> "{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ATLAS-3289) Add a configuration to population dynamic attributes. If the config is disabled, read the value from the Hook
[ https://issues.apache.org/jira/browse/ATLAS-3289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang reassigned ATLAS-3289: --- Assignee: Merryle Wang > Add a configuration to population dynamic attributes. If the config is > disabled, read the value from the Hook > - > > Key: ATLAS-3289 > URL: https://issues.apache.org/jira/browse/ATLAS-3289 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-3287) Populate Dynamic Relationship Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang reassigned ATLAS-3287: --- Assignee: Merryle Wang > Populate Dynamic Relationship Attributes > > > Key: ATLAS-3287 > URL: https://issues.apache.org/jira/browse/ATLAS-3287 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 336h > Remaining Estimate: 336h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3287) Populate Dynamic Relationship Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang updated ATLAS-3287: Remaining Estimate: 336h Original Estimate: 336h > Populate Dynamic Relationship Attributes > > > Key: ATLAS-3287 > URL: https://issues.apache.org/jira/browse/ATLAS-3287 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Priority: Major > Original Estimate: 336h > Remaining Estimate: 336h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3288) Implement the DynamicAttributesEvaluator method
[ https://issues.apache.org/jira/browse/ATLAS-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang updated ATLAS-3288: Remaining Estimate: 672h Original Estimate: 672h > Implement the DynamicAttributesEvaluator method > > > Key: ATLAS-3288 > URL: https://issues.apache.org/jira/browse/ATLAS-3288 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Priority: Major > Original Estimate: 672h > Remaining Estimate: 672h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3288) Implement the DynamicAttributesEvaluator method
Aadarsh Jajodia created ATLAS-3288: -- Summary: Implement the DynamicAttributesEvaluator method Key: ATLAS-3288 URL: https://issues.apache.org/jira/browse/ATLAS-3288 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Description: We aim to implement the following methods in this Jira AtlasAttributes isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db For AtlasEntityTypes - getDynEvalAttributes() was: We aim to implement the following methods in this Jira AtlasAttributes isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db For AtlasEntityTypes - getDynEvalAttributes() dynValueEval(AtlasEntity entity) - Takes in the entity instance and evaluates the dynamic attributes for that entity > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 168h > Remaining Estimate: 168h > > We aim to implement the following methods in this Jira > AtlasAttributes > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > For AtlasEntityTypes - > getDynEvalAttributes() -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3287) Populate Dynamic Relationship Attributes
Aadarsh Jajodia created ATLAS-3287: -- Summary: Populate Dynamic Relationship Attributes Key: ATLAS-3287 URL: https://issues.apache.org/jira/browse/ATLAS-3287 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Description: We aim to implement the following methods in this Jira AtlasAttributes isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db For AtlasEntityTypes - getDynEvalAttributes() dynValueEval(AtlasEntity entity) - Takes in the entity instance and evaluates the dynamic attributes for that entity was: We aim to implement the following methods in this Jira isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 168h > Remaining Estimate: 168h > > We aim to implement the following methods in this Jira > AtlasAttributes > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > For AtlasEntityTypes - > getDynEvalAttributes() > dynValueEval(AtlasEntity entity) - Takes in the entity instance and evaluates > the dynamic attributes for that entity -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang updated ATLAS-3286: Remaining Estimate: 168h Original Estimate: 168h > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 168h > Remaining Estimate: 168h > > We aim to implement the following methods in this Jira > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Description: We aim to implement the following methods in this Jira isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Priority: Major > > We aim to implement the following methods in this Jira > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang reassigned ATLAS-3286: --- Assignee: Merryle Wang > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > > We aim to implement the following methods in this Jira > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Summary: Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes (was: Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType) > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType
Aadarsh Jajodia created ATLAS-3286: -- Summary: Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType Key: ATLAS-3286 URL: https://issues.apache.org/jira/browse/ATLAS-3286 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
[
https://issues.apache.org/jira/browse/ATLAS-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aadarsh Jajodia updated ATLAS-3285:
---
Description:
An example of a Dynamic Attribute for a Hive Column looks something like this
in the hive models file
{code:java}
"options": {
"dynAttribute:qualifiedName":
"{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
}
{code}
was:
An example of a Dynamic Attribute for a Hive Column something like this
{code:java}
"options": {
"dynAttribute:qualifiedName":
"{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
}
{code}
> Defining the template version for QualifiedName in the models file for hive
> entity types
>
>
> Key: ATLAS-3285
> URL: https://issues.apache.org/jira/browse/ATLAS-3285
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Aadarsh Jajodia
>Assignee: Merryle Wang
>Priority: Major
>
> An example of a Dynamic Attribute for a Hive Column looks something like this
> in the hive models file
> {code:java}
> "options": {
> "dynAttribute:qualifiedName":
> "{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
[
https://issues.apache.org/jira/browse/ATLAS-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aadarsh Jajodia updated ATLAS-3285:
---
Description:
An example of a Dynamic Attribute for a Hive Column something like this
{code:java}
"options": {
"dynAttribute:qualifiedName":
"{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
}
{code}
> Defining the template version for QualifiedName in the models file for hive
> entity types
>
>
> Key: ATLAS-3285
> URL: https://issues.apache.org/jira/browse/ATLAS-3285
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Aadarsh Jajodia
>Assignee: Merryle Wang
>Priority: Major
>
> An example of a Dynamic Attribute for a Hive Column something like this
> {code:java}
> "options": {
> "dynAttribute:qualifiedName":
> "{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
[ https://issues.apache.org/jira/browse/ATLAS-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang reassigned ATLAS-3285: --- Assignee: Merryle Wang > Defining the template version for QualifiedName in the models file for hive > entity types > > > Key: ATLAS-3285 > URL: https://issues.apache.org/jira/browse/ATLAS-3285 > Project: Atlas > Issue Type: Sub-task >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
Aadarsh Jajodia created ATLAS-3285: -- Summary: Defining the template version for QualifiedName in the models file for hive entity types Key: ATLAS-3285 URL: https://issues.apache.org/jira/browse/ATLAS-3285 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-3284) Adding functionality to compute dynamic attributes for an entity.
[ https://issues.apache.org/jira/browse/ATLAS-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Merryle Wang reassigned ATLAS-3284: --- Assignee: Merryle Wang > Adding functionality to compute dynamic attributes for an entity. > - > > Key: ATLAS-3284 > URL: https://issues.apache.org/jira/browse/ATLAS-3284 > Project: Atlas > Issue Type: New Feature >Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > > This task aims at computing dynamic attributes for an entity. The first one > we are going to take up in qualifiedName -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3284) Adding functionality to compute dynamic attributes for an entity.
Aadarsh Jajodia created ATLAS-3284: -- Summary: Adding functionality to compute dynamic attributes for an entity. Key: ATLAS-3284 URL: https://issues.apache.org/jira/browse/ATLAS-3284 Project: Atlas Issue Type: New Feature Reporter: Aadarsh Jajodia This task aims at computing dynamic attributes for an entity. The first one we are going to take up in qualifiedName -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3279) Performance regression after recent updates to disable full-text index
[ https://issues.apache.org/jira/browse/ATLAS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866058#comment-16866058 ] ASF subversion and git services commented on ATLAS-3279: Commit 59648d289ee8a8c36c1f8e4ba7f3807418a64639 in atlas's branch refs/heads/master from Madhan Neethiraj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=59648d2 ] ATLAS-3279: avoid unncessary retrieval of entity-extended info while sending notifications Change-Id: I82e0bba27010709c74cd98a93f8a9c617577535e > Performance regression after recent updates to disable full-text index > -- > > Key: ATLAS-3279 > URL: https://issues.apache.org/jira/browse/ATLAS-3279 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 2.1.0, 3.0.0 > > Attachments: ATLAS-3279.patch > > > With recent search enhancements in ATLAS-3100 and ATLAS-3246, full-text index > is no more needed; and ATLAS-3262 disabled populating full-text index. After > this change, Atlas server takes a longer time to send entity notifications; > the cache used by the notification module was earlier populated while > computing full-text index string; now, this is done by > AtlasInstanceConverter, which retrieves entity-with-extended-info (to be used > only to send V1 style notifications), which resulted in performance > regression in processing notifications. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70864: ATLAS-3279: avoid unncessary retrieval of entity-extended info while sending notifications
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70864/#review215927 --- Ship it! Ship It! - Sarath Subramanian On June 17, 2019, 1:41 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70864/ > --- > > (Updated June 17, 2019, 1:41 a.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, keval bhatt, > Sridhar K, Le Ma, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon > Rodrigues, Saqeeb Shaikh, and Sarath Subramanian. > > > Bugs: ATLAS-3279 > https://issues.apache.org/jira/browse/ATLAS-3279 > > > Repository: atlas > > > Description > --- > > updated entity-notification module to avoid using AtlasInstanceConverter > where feasible, to avoid retrieval of entity-extended-info > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapperV2.java > caa660491 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java > 2e47a50c0 > > > Diff: https://reviews.apache.org/r/70864/diff/1/ > > > Testing > --- > > - pre-commit tests run: > https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1209 > > > Thanks, > > Madhan Neethiraj > >
[jira] [Commented] (ATLAS-3282) UI : use search suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865960#comment-16865960 ] ASF subversion and git services commented on ATLAS-3282: Commit 9062e2c861628d227f0c2616c5c84f2f90af9214 in atlas's branch refs/heads/master from Keval Bhatt [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=9062e2c ] ATLAS-3282 : UI : use search suggestions API for quick-search Signed-off-by: Sarath Subramanian > UI : use search suggestions API for quick-search > > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > Attachments: ATLAS-3282.patch > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (ATLAS-3198) update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements
[ https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-3198. --- Resolution: Fixed > update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements > - > > Key: ATLAS-3198 > URL: https://issues.apache.org/jira/browse/ATLAS-3198 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Umesh Padashetty >Assignee: Yu-Hsin Chang >Priority: Critical > Fix For: trunk > > Attachments: ATLAS-3198.patch, ATLAS_HOOK and ATLAS_ENTITIES > messages.xlsx > > > With the recent changes, we have now stopped ingesting simple insert > operations since with this operation, there is no change in the metadata. > Ideally, we should be doing similar improvements w.r.t. update, delete and > truncate operations too. > Currently, for update/delete/truncate, we are pushing messages in both > ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided. > If we stop ingesting updates/deletes/truncates, it will further help us > reduce the burden on Kafka and save CPU cycles > I have created and attached an excel sheet, with the list of operations and > the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic, > along with suitable comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3198) update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements
[ https://issues.apache.org/jira/browse/ATLAS-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865889#comment-16865889 ] ASF subversion and git services commented on ATLAS-3198: Commit 4e1eeb4cdcd8bc0601564434176eb200b2811c2e in atlas's branch refs/heads/master from Yu-Hsin Shih [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=4e1eeb4 ] ATLAS-3198: Stop pushing message to ATLAS_HOOK for Truncate Table/Delete data/Update data Both ATLAS_HOOK and ATLAS_ENTITIES should stop receiving message from Truncate Table/Delete data/Update data Signed-off-by: Sarath Subramanian > update/delete/truncate operations ATLAS_HOOK and ATLAS_ENTITIES improvements > - > > Key: ATLAS-3198 > URL: https://issues.apache.org/jira/browse/ATLAS-3198 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Umesh Padashetty >Assignee: Yu-Hsin Chang >Priority: Critical > Fix For: trunk > > Attachments: ATLAS-3198.patch, ATLAS_HOOK and ATLAS_ENTITIES > messages.xlsx > > > With the recent changes, we have now stopped ingesting simple insert > operations since with this operation, there is no change in the metadata. > Ideally, we should be doing similar improvements w.r.t. update, delete and > truncate operations too. > Currently, for update/delete/truncate, we are pushing messages in both > ATLAS_HOOK as well as ATLAS_ENTITIES topic, which can be avoided. > If we stop ingesting updates/deletes/truncates, it will further help us > reduce the burden on Kafka and save CPU cycles > I have created and attached an excel sheet, with the list of operations and > the corresponding messages generated in ATLAS_HOOK and ATLAS_ENTITIES topic, > along with suitable comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70784: ATLAS-3198: Stop pushing message to ATLAS_HOOK for Truncate Table/Delete data/Update data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70784/#review215924 --- Ship it! Ship It! - Sarath Subramanian On June 4, 2019, 11:41 a.m., Yu-Hsin Shih wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70784/ > --- > > (Updated June 4, 2019, 11:41 a.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3198 > https://issues.apache.org/jira/browse/ATLAS-3198 > > > Repository: atlas > > > Description > --- > > Stop pushing message to ATLAS_HOOK for Truncate Table/Delete data/Update > data, make sure both ATLAS_HOOK and ATLAS_ENTITIES no longer receive message > to save CPU time. > > > Diffs > - > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java > c0094c634 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java > c4b3ac72f > addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java > c83016291 > > > Diff: https://reviews.apache.org/r/70784/diff/1/ > > > Testing > --- > > 1. Manual tested >Run Truncate Table/Delete data/Update data on hive via beeline and > validate in a VM with hive and atlas service running. After code change, > message is no longer recorded for these operations in both ATLAS_HOOK and > ATLAS_ENTITIES. > 2. Update Integration Test for Truncate Table > > > Thanks, > > Yu-Hsin Shih > >
[DRAFT] Board report for Apache Atlas: June 2019
Atlas team, Please review the draft board report below and send your feedback/comments. Thanks, Madhan ## Description: Apache Atlas is a scalable and extensible set of core foundational governance services that enables enterprises to effectively and efficiently meet their compliance requirements within Hadoop and allows integration with the complete enterprise data ecosystem ## Issues: There are no issues requiring board attention at this time. ## Activity: - Released 2.0.0 major version, to support Hadoop 3, HBase 2, Solr 7, Kafka 2, Hive 3 - Released 1.2.0 maintenance version, with 170 fixes/enhancements - added Atlas hook for Hive Meta Store (HMS) - added Atlas hook for Apache Impala - added framework to track status of typedef and Java patches - added a REST endpoint and UI to publish runtime stats from Atlas server - UI improvements in rendering relationships - updated QuickStart tutorial to replace use of object-ref attributes with relationships - updated RDBMS types to replace use of object-ref attributes with relationships, for better performance - added type-definitions to capture metadata and lineage from Spark - enhanced Hive hook to capture DDL statements - working on quick-search functionality, to provide a simpler search interface - working on 0.8.4 maintenance release, likely to be the last release for 0.8 line ## Health report: - 12 new contributors added in last 3 months: Carol Drummond, Chaitali Borol, Diego Marino Monetti, Kirankumar D G, Li Na, Mandar Ambawan, Mayank Jain, Merryle Wang, Rahul Kurup, Umesh S. Padashetty, Xinran Yu, Yu-Hsin Shih ## PMC changes: - Currently 34 PMC members - 1 new PMC member added in last 3 months, on 5/13/2019 - Last addition to PMC role was Ashutosh Mestry ## Committer base changes: - Currently 38 committers - No new committers added in last 3 months - Last addition to committer role was on 10/15/2018 ## Releases: 0.8.4plan to release by 06/31/2019 1.2.0was released on 06/12/2019 2.0.0was released on 05/14/2019 0.8.3was released on 10/31/2018 1.1.0was released on 09/17/2018 1.0.0was released on 06/02/2018 0.8.2was released on 02/05/2018 1.0.0-alpha was released on 01/25/2018 0.8.1was released on 08/29/2017 0.8-incubating was released on 03/16/2017 0.7.1-incubating was released on 01/26/2017 0.7-incubating was released on 07/09/2016 0.6-incubating was released on 12/31/2015 0.5-incubating was released on 07/11/2015
[jira] [Commented] (ATLAS-3282) UI : use search suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865719#comment-16865719 ] Sarath Subramanian commented on ATLAS-3282: --- Thanks for the patch [~kevalbhatt]. +1 > UI : use search suggestions API for quick-search > > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > Attachments: ATLAS-3282.patch > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3281) Allow excluding lineage related to deleted entities
[
https://issues.apache.org/jira/browse/ATLAS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3281:
--
Description:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is displayed as
follow:
!screenshot-1.png!
When {{excludeDeletedEndpoints}} is set to true, the lineage is displayed as
follow:
!screenshot-2.png!
was:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
!screenshot-1.png!
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
!screenshot-2.png!
> Allow excluding lineage related to deleted entities
> ---
>
> Key: ATLAS-3281
> URL: https://issues.apache.org/jira/browse/ATLAS-3281
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: ATLAS-3281.patch, screenshot-1.png, screenshot-2.png
>
>
> If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}}
> will still return hive_process related to the table. But in some scenarios,
> we don't want to display hive_process related to deleted entities in web UI.
> So we propose to add a parameter {{excludeDeletedEndpoints}} to
> {{getLineageQuery.getLineageQuery}}. When set to true, the function will
> exclude lineages related to deleted entities. We also add the parameter to
> {{AtlasLineageService}},
> {{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
> {{LineageREST}} accordingly.
> When {{excludeDeletedEndpoints}} is set to false, the lineage is displayed as
> follow:
> !screenshot-1.png!
> When {{excludeDeletedEndpoints}} is set to true, the lineage is displayed as
> follow:
> !screenshot-2.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: alter_lineage.jpg
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.jpg, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
>
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!alter_lineage.jpg!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.jpg, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.jpg!
>
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3283) Export-import UTs are getting skipped
[ https://issues.apache.org/jira/browse/ATLAS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Bonte updated ATLAS-3283: Attachment: ATLAS-3283-Export-import-UTs-are-getting-skipped.patch > Export-import UTs are getting skipped > - > > Key: ATLAS-3283 > URL: https://issues.apache.org/jira/browse/ATLAS-3283 > Project: Atlas > Issue Type: Test >Reporter: Nikhil Bonte >Assignee: Nikhil Bonte >Priority: Major > Attachments: ATLAS-3283-Export-import-UTs-are-getting-skipped.patch > > > Some unit test cases for import & export APIs are getting skipped & are not > executed completely. > Though this do not interrupt build process, but need to be fixed as it is > simply skipping UTs which could lead having some problems in import & export > APIs unnoticed. > This jira fixes skipping of these UTs. > Classes having skipping UTs: > * ExportSkipLineageTest - all > * ExportIncrementalTest - all > * IncrementalExportEntityProviderTest - all > * ImportServiceTest - 2 of 20 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3283) Export-import UTs are getting skipped
Nikhil Bonte created ATLAS-3283: --- Summary: Export-import UTs are getting skipped Key: ATLAS-3283 URL: https://issues.apache.org/jira/browse/ATLAS-3283 Project: Atlas Issue Type: Test Reporter: Nikhil Bonte Assignee: Nikhil Bonte Some unit test cases for import & export APIs are getting skipped & are not executed completely. Though this do not interrupt build process, but need to be fixed as it is simply skipping UTs which could lead having some problems in import & export APIs unnoticed. This jira fixes skipping of these UTs. Classes having skipping UTs: * ExportSkipLineageTest - all * ExportIncrementalTest - all * IncrementalExportEntityProviderTest - all * ImportServiceTest - 2 of 20 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) Ui : use search suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Summary: Ui : use search suggestions API for quick-search (was: Ui : User suggestions API for quick-search) > Ui : use search suggestions API for quick-search > > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > Attachments: ATLAS-3282.patch > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) UI : use search suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Summary: UI : use search suggestions API for quick-search (was: Ui : use search suggestions API for quick-search) > UI : use search suggestions API for quick-search > > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > Attachments: ATLAS-3282.patch > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) UI : use search suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Attachment: ATLAS-3282.patch > UI : use search suggestions API for quick-search > > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > Attachments: ATLAS-3282.patch > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3263) UI: Assign button (Create Term) should be blocked while an API call is in progress
[ https://issues.apache.org/jira/browse/ATLAS-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865567#comment-16865567 ] ASF subversion and git services commented on ATLAS-3263: Commit 0edb78b660b8559c2d8d4d20e7b03a05cb4deda9 in atlas's branch refs/heads/branch-1.0 from gutkaBinit [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0edb78b ] ATLAS-3263 UI: Assign button (Create Term) should be blocked while an API call is in progress Signed-off-by: kevalbhatt (cherry picked from commit 3f67eb3cd73d445b5d35b29df697564ecf32a345) > UI: Assign button (Create Term) should be blocked while an API call is in > progress > -- > > Key: ATLAS-3263 > URL: https://issues.apache.org/jira/browse/ATLAS-3263 > Project: Atlas > Issue Type: Bug >Reporter: Rahul Kurup >Assignee: Binit Gutka >Priority: Minor > Fix For: 1.2.0, 2.1.0, 3.0.0 > > Attachments: ATLAS-3263.patch, disableassignbutton.jpg > > > It is possible that if the user clicks the Assign button in the Create Term > window multiple times rapidly, the UI detects and acknowledges the same as > displayed in the screenshot below. > !disableassignbutton.jpg|width=200,height=200! > The side-effect of this behaviour is that these clicks are recognized in the > Audit Log of the entity (to whom term is assigned) and we have multiple "Term > Created" entries even though it refers to only one term being created. > To avoid such behaviour, it would be better to lock the Assign button while > the API call is in progress. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3263) UI: Assign button (Create Term) should be blocked while an API call is in progress
[ https://issues.apache.org/jira/browse/ATLAS-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865565#comment-16865565 ] ASF subversion and git services commented on ATLAS-3263: Commit 620bc29ccd861f59ef8650197ccb078a552a8c1a in atlas's branch refs/heads/branch-2.0 from gutkaBinit [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=620bc29 ] ATLAS-3263 UI: Assign button (Create Term) should be blocked while an API call is in progress Signed-off-by: kevalbhatt (cherry picked from commit 3f67eb3cd73d445b5d35b29df697564ecf32a345) > UI: Assign button (Create Term) should be blocked while an API call is in > progress > -- > > Key: ATLAS-3263 > URL: https://issues.apache.org/jira/browse/ATLAS-3263 > Project: Atlas > Issue Type: Bug >Reporter: Rahul Kurup >Assignee: Binit Gutka >Priority: Minor > Attachments: ATLAS-3263.patch, disableassignbutton.jpg > > > It is possible that if the user clicks the Assign button in the Create Term > window multiple times rapidly, the UI detects and acknowledges the same as > displayed in the screenshot below. > !disableassignbutton.jpg|width=200,height=200! > The side-effect of this behaviour is that these clicks are recognized in the > Audit Log of the entity (to whom term is assigned) and we have multiple "Term > Created" entries even though it refers to only one term being created. > To avoid such behaviour, it would be better to lock the Assign button while > the API call is in progress. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3263) UI: Assign button (Create Term) should be blocked while an API call is in progress
[ https://issues.apache.org/jira/browse/ATLAS-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3263: --- Fix Version/s: 3.0.0 2.1.0 1.2.0 > UI: Assign button (Create Term) should be blocked while an API call is in > progress > -- > > Key: ATLAS-3263 > URL: https://issues.apache.org/jira/browse/ATLAS-3263 > Project: Atlas > Issue Type: Bug >Reporter: Rahul Kurup >Assignee: Binit Gutka >Priority: Minor > Fix For: 1.2.0, 2.1.0, 3.0.0 > > Attachments: ATLAS-3263.patch, disableassignbutton.jpg > > > It is possible that if the user clicks the Assign button in the Create Term > window multiple times rapidly, the UI detects and acknowledges the same as > displayed in the screenshot below. > !disableassignbutton.jpg|width=200,height=200! > The side-effect of this behaviour is that these clicks are recognized in the > Audit Log of the entity (to whom term is assigned) and we have multiple "Term > Created" entries even though it refers to only one term being created. > To avoid such behaviour, it would be better to lock the Assign button while > the API call is in progress. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3263) UI: Assign button (Create Term) should be blocked while an API call is in progress
[ https://issues.apache.org/jira/browse/ATLAS-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865564#comment-16865564 ] ASF subversion and git services commented on ATLAS-3263: Commit 3f67eb3cd73d445b5d35b29df697564ecf32a345 in atlas's branch refs/heads/master from gutkaBinit [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=3f67eb3 ] ATLAS-3263 UI: Assign button (Create Term) should be blocked while an API call is in progress Signed-off-by: kevalbhatt > UI: Assign button (Create Term) should be blocked while an API call is in > progress > -- > > Key: ATLAS-3263 > URL: https://issues.apache.org/jira/browse/ATLAS-3263 > Project: Atlas > Issue Type: Bug >Reporter: Rahul Kurup >Assignee: Binit Gutka >Priority: Minor > Attachments: ATLAS-3263.patch, disableassignbutton.jpg > > > It is possible that if the user clicks the Assign button in the Create Term > window multiple times rapidly, the UI detects and acknowledges the same as > displayed in the screenshot below. > !disableassignbutton.jpg|width=200,height=200! > The side-effect of this behaviour is that these clicks are recognized in the > Audit Log of the entity (to whom term is assigned) and we have multiple "Term > Created" entries even though it refers to only one term being created. > To avoid such behaviour, it would be better to lock the Assign button while > the API call is in progress. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) Ui : User suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Issue Type: Improvement (was: Bug) > Ui : User suggestions API for quick-search > -- > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) Ui : User suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Fix Version/s: (was: 2.1.0) > Ui : User suggestions API for quick-search > -- > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) Ui : User suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Affects Version/s: 3.0.0 > Ui : User suggestions API for quick-search > -- > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 3.0.0 > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3282) Ui : User suggestions API for quick-search
Keval Bhatt created ATLAS-3282: -- Summary: Ui : User suggestions API for quick-search Key: ATLAS-3282 URL: https://issues.apache.org/jira/browse/ATLAS-3282 Project: Atlas Issue Type: Bug Reporter: Keval Bhatt Assignee: Keval Bhatt Use quick & suggestions API for quick-search API details: http://localhost:21000/api/atlas/v2/search/quick?query=sal* http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal For more details regarding these API please check [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3282) Ui : User suggestions API for quick-search
[ https://issues.apache.org/jira/browse/ATLAS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keval Bhatt updated ATLAS-3282: --- Fix Version/s: 3.0.0 2.1.0 > Ui : User suggestions API for quick-search > -- > > Key: ATLAS-3282 > URL: https://issues.apache.org/jira/browse/ATLAS-3282 > Project: Atlas > Issue Type: Bug >Reporter: Keval Bhatt >Assignee: Keval Bhatt >Priority: Major > Fix For: 2.1.0, 3.0.0 > > > Use quick & suggestions API for quick-search > > API details: > http://localhost:21000/api/atlas/v2/search/quick?query=sal* > http://localhost:21000/api/atlas/v2/search/suggestions?prefixString=sal > For more details regarding these API please check > [ATLAS-3270|https://issues.apache.org/jira/browse/ATLAS-3270] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
>
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: alter_lineage.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: 2.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!alter_lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: lineage.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: test.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!alter_lineage.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!test.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.png, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !alter_lineage.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: alter_lineage.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, alter_lineage.png, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !test.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: 5.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !test.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!test.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !test.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: test.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, test.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: screenshot-1.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
!5.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> (ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
> !5.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: 5.png
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, 5.png, screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> (ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows
(ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
!screenshot-1.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows
> (ALTERTABLE_ADDCOLS:sample.test_external@bd:1560757768000->:sample.test_external@bd:1560757768000)：
> !screenshot-1.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3281) Allow excluding lineage related to deleted entities
[
https://issues.apache.org/jira/browse/ATLAS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3281:
--
Description:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
!screenshot-1.png!
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
!screenshot-2.png!
was:
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
> Allow excluding lineage related to deleted entities
> ---
>
> Key: ATLAS-3281
> URL: https://issues.apache.org/jira/browse/ATLAS-3281
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}}
> will still return hive_process related to the table. But in some scenarios,
> we don't want to display hive_process related to deleted entities in web UI.
> So we propose to add a parameter {{excludeDeletedEndpoints}} to
> {{getLineageQuery.getLineageQuery}}. When set to true, the function will
> exclude lineages related to deleted entities. We also add the parameter to
> {{AtlasLineageService}},
> {{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
> {{LineageREST}} accordingly.
> When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
> follow:
> !screenshot-1.png!
> When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
> follow:
> !screenshot-2.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3281) Allow excluding lineage related to deleted entities
[
https://issues.apache.org/jira/browse/ATLAS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3281:
--
Attachment: screenshot-1.png
> Allow excluding lineage related to deleted entities
> ---
>
> Key: ATLAS-3281
> URL: https://issues.apache.org/jira/browse/ATLAS-3281
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 1.1.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.1.0
>
> Attachments: screenshot-1.png, screenshot-2.png
>
>
> If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}}
> will still return hive_process related to the table. But in some scenarios,
> we don't want to display hive_process related to deleted entities in web UI.
> So we propose to add a parameter {{excludeDeletedEndpoints}} to
> {{getLineageQuery.getLineageQuery}}. When set to true, the function will
> exclude lineages related to deleted entities. We also add the parameter to
> {{AtlasLineageService}},
> {{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
> {{LineageREST}} accordingly.
> When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
> follow:
> When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
> follow:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3281) Allow excluding lineage related to deleted entities
Jiaqi Shan created ATLAS-3281:
-
Summary: Allow excluding lineage related to deleted entities
Key: ATLAS-3281
URL: https://issues.apache.org/jira/browse/ATLAS-3281
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Affects Versions: 1.1.0
Reporter: Jiaqi Shan
Fix For: 1.1.0
If a hive_table was deleted, {{EntityLineageService.getAtlasLineageInfo}} will
still return hive_process related to the table. But in some scenarios, we don't
want to display hive_process related to deleted entities in web UI. So we
propose to add a parameter {{excludeDeletedEndpoints}} to
{{getLineageQuery.getLineageQuery}}. When set to true, the function will
exclude lineages related to deleted entities. We also add the parameter to
{{AtlasLineageService}},
{{AtlasGremlinQueryProvider}},{{AtlasGremlin3QueryProvider}} and
{{LineageREST}} accordingly.
When {{excludeDeletedEndpoints}} is set to false, the lineage is display as
follow:
When {{excludeDeletedEndpoints}} is set to true, the lineage is display as
follow:
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: 0001-ATLAS-3280.patch
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: 0001-ATLAS-3280.patch, screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Attachment: (was: image-2019-06-17-16-50-07-833.png)
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!screenshot-1.png! !image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: screenshot-1.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !screenshot-1.png!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image.jpg|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: image-2019-06-17-16-50-07-833.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !image-2019-06-17-16-50-07-833.png|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image.jpg|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
[^image-2019-06-17-16-50-07-833.png]
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: image-2019-06-17-16-50-07-833.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> !image.jpg|thumbnail!
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
[
https://issues.apache.org/jira/browse/ATLAS-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiaqi Shan updated ATLAS-3280:
--
Description:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
[^image-2019-06-17-16-50-07-833.png]
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
was:
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
> Hive Hook creates lineage incorrectly when alter external table or view
> ---
>
> Key: ATLAS-3280
> URL: https://issues.apache.org/jira/browse/ATLAS-3280
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 1.0.0, 1.1.0, 2.0.0
>Reporter: Jiaqi Shan
>Priority: Major
> Fix For: 1.0.0, 1.1.0, 2.0.0
>
> Attachments: image-2019-06-17-16-50-07-833.png
>
>
> When altering an external table or a view, Hive Hook will create lineage
> incorrectly as follows：
> [^image-2019-06-17-16-50-07-833.png]
> So we propose to add a parameter {{ignoreProcessEntity}} to
> {{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage
> to external table or view. We also add the parameter to {{AlterTable}}
> accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3280) Hive Hook creates lineage incorrectly when alter external table or view
Jiaqi Shan created ATLAS-3280:
-
Summary: Hive Hook creates lineage incorrectly when alter external
table or view
Key: ATLAS-3280
URL: https://issues.apache.org/jira/browse/ATLAS-3280
Project: Atlas
Issue Type: Bug
Components: atlas-core
Affects Versions: 2.0.0, 1.1.0, 1.0.0
Reporter: Jiaqi Shan
Fix For: 2.0.0, 1.1.0, 1.0.0
Attachments: image-2019-06-17-16-50-07-833.png
When altering an external table or a view, Hive Hook will create lineage
incorrectly as follows：
!image-2019-06-17-16-50-07-833.png|thumbnail!
So we propose to add a parameter {{ignoreProcessEntity}} to
{{CreateTable.getEntites}}. When set to true, Hive Hook won’t create lineage to
external table or view. We also add the parameter to {{AlterTable}} accordingly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Review Request 70864: ATLAS-3279: avoid unncessary retrieval of entity-extended info while sending notifications
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70864/ --- Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, keval bhatt, Sridhar K, Le Ma, Mandar Ambawane, mayank jain, Nikhil Bonte, Nixon Rodrigues, Saqeeb Shaikh, and Sarath Subramanian. Bugs: ATLAS-3279 https://issues.apache.org/jira/browse/ATLAS-3279 Repository: atlas Description --- updated entity-notification module to avoid using AtlasInstanceConverter where feasible, to avoid retrieval of entity-extended-info Diffs - repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapperV2.java caa660491 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java 2e47a50c0 Diff: https://reviews.apache.org/r/70864/diff/1/ Testing --- - pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1209 Thanks, Madhan Neethiraj
[jira] [Commented] (ATLAS-3036) Improve FullTextMapper performance during entity retrieval
[ https://issues.apache.org/jira/browse/ATLAS-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865390#comment-16865390 ] ASF subversion and git services commented on ATLAS-3036: Commit 0eeaae6992ef2dac9c458a68bdcdfb869256e184 in atlas's branch refs/heads/branch-1.0 from Sarath Subramanian [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0eeaae6 ] ATLAS-3036: Improve FullTextMapper performance during entity retrieval (cherry picked from commit a08a5a39a83fcfee0965471a3a1b1c29279feea3) (cherry picked from commit 7ff396af047c8ff829dbe5b8a4094177fb3e9686) (cherry picked from commit 2fdbc851db06f7ebe96cc3fde4d8507d088e7ec6) (cherry picked from commit 66c2964e05f8421d8dfb395f1f0cf61d1f33a8e4) > Improve FullTextMapper performance during entity retrieval > -- > > Key: ATLAS-3036 > URL: https://issues.apache.org/jira/browse/ATLAS-3036 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 0.8.3 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 0.8.4 > > Attachments: > 0001-ATLAS-3036-Improve-FullTextMapper-performance-during-MASTER.patch, > 0001-ATLAS-3036-Improve-FullTextMapper-performance-during.patch > > > Improve FullTextMapper performance during entity retrieval -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3036) Improve FullTextMapper performance during entity retrieval
[ https://issues.apache.org/jira/browse/ATLAS-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865385#comment-16865385 ] ASF subversion and git services commented on ATLAS-3036: Commit 075f331b74a08c818e7030183288a54bcdf35487 in atlas's branch refs/heads/branch-2.0 from Sarath Subramanian [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=075f331 ] ATLAS-3036: Improve FullTextMapper performance during entity retrieval (cherry picked from commit a08a5a39a83fcfee0965471a3a1b1c29279feea3) (cherry picked from commit 7ff396af047c8ff829dbe5b8a4094177fb3e9686) (cherry picked from commit 2fdbc851db06f7ebe96cc3fde4d8507d088e7ec6) (cherry picked from commit 66c2964e05f8421d8dfb395f1f0cf61d1f33a8e4) > Improve FullTextMapper performance during entity retrieval > -- > > Key: ATLAS-3036 > URL: https://issues.apache.org/jira/browse/ATLAS-3036 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 0.8.3 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 0.8.4 > > Attachments: > 0001-ATLAS-3036-Improve-FullTextMapper-performance-during-MASTER.patch, > 0001-ATLAS-3036-Improve-FullTextMapper-performance-during.patch > > > Improve FullTextMapper performance during entity retrieval -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3036) Improve FullTextMapper performance during entity retrieval
[ https://issues.apache.org/jira/browse/ATLAS-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865384#comment-16865384 ] ASF subversion and git services commented on ATLAS-3036: Commit 66c2964e05f8421d8dfb395f1f0cf61d1f33a8e4 in atlas's branch refs/heads/master from Sarath Subramanian [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=66c2964 ] ATLAS-3036: Improve FullTextMapper performance during entity retrieval (cherry picked from commit a08a5a39a83fcfee0965471a3a1b1c29279feea3) (cherry picked from commit 7ff396af047c8ff829dbe5b8a4094177fb3e9686) (cherry picked from commit 2fdbc851db06f7ebe96cc3fde4d8507d088e7ec6) > Improve FullTextMapper performance during entity retrieval > -- > > Key: ATLAS-3036 > URL: https://issues.apache.org/jira/browse/ATLAS-3036 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 0.8.3 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 0.8.4 > > Attachments: > 0001-ATLAS-3036-Improve-FullTextMapper-performance-during-MASTER.patch, > 0001-ATLAS-3036-Improve-FullTextMapper-performance-during.patch > > > Improve FullTextMapper performance during entity retrieval -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3277) Add default atlas conf path in the import-* scripts
[ https://issues.apache.org/jira/browse/ATLAS-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16865352#comment-16865352 ] Nixon Rodrigues commented on ATLAS-3277: [~vishal.suvagia], Thanks for the patch, the changes looks good. +1 for the patch. > Add default atlas conf path in the import-* scripts > --- > > Key: ATLAS-3277 > URL: https://issues.apache.org/jira/browse/ATLAS-3277 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Suvagia >Assignee: Vishal Suvagia >Priority: Minor > Attachments: ATLAS-3277.patch > > > Update Atlas import-* scripts to add default atlas conf path, in case if > atlas-application.properties is not available in the hook component-conf path -- This message was sent by Atlassian JIRA (v7.6.3#76005)
