Re: Review Request 71969: ATLAS-3486: introduce Namespace typedef
> On Jan. 8, 2020, 11:25 p.m., Sarath Subramanian wrote: > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java > > Lines 303 (patched) > > <https://reviews.apache.org/r/71969/diff/2/?file=2199349#file2199349line303> > > > > 'typeDefStore' is class variable. not needed to be passed in method > > params. please review. All of the methods in AtlasStructDefStoreV2 take in the typeDefStore as a parameter. Hence passing this. - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71969/#review219184 ------- On Jan. 8, 2020, 1:23 a.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71969/ > --- > > (Updated Jan. 8, 2020, 1:23 a.m.) > > > Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan > Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3486: introduce Namespace typedef > > > Diffs > - > > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java > 3ebd70d981a734035416fdf9d880cebd68fe2f30 > intg/src/main/java/org/apache/atlas/model/TypeCategory.java > f06f64f450f407e3f9a0e742726ff4dd12ccc695 > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de > intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java > 3634fdfd313639eb97b3c4698e091487b0e44a80 > intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java > 4ee68a936f99bb4c819b5335da2cc8bf7d539397 > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 4742d1c0ea7d7a24fee0453d6fe052579ba6bbd2 > intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/type/AtlasStructType.java > 3475ce6aa8dd4cad4caf08ab5501fc8dc37c1339 > intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java > b071dc9d664cee9e1ffc54726ffbf15f4f602d30 > intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java > 0883d54f490e22c6510e6fc0cb804b87713a7ecb > intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java > dba2d88146eff314191ae6bb24ad7337b0ea10ae > intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java > 02613b5f7250b14324ed294c22de079b74d55b08 > > intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java > 7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 > > repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java > 6016723d82abd47fa15c5f3a94b70c3ce0673413 > > repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java > 5bad615cd5c223b8246d8443ee5b33925a8b22e9 > > repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java > 08b00e76a5a96aa898f4fc252d3abc22845be50b > > repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java > b04f188beff345d417141d194f0a61e584352111 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java > a5ccfb5b2055c88f596312f4033bc0034d3d165c > > repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java > 6cd0ee331b7ae24757b58e76ec47bf556106846a > webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java > fb56fad6412079e20dd3e345b81a08d9e5ace657 > > > Diff: https://reviews.apache.org/r/71969/diff/2/ > > > Testing > --- > > As suggested by Madhan in this review here > https://reviews.apache.org/r/71898/, it makes sense to have AtlasNamespaceDef > inherit from AtlasStructDef. This change builds on top of that > > > Thanks, > > Aadarsh Jajodia > >
Re: Review Request 71969: ATLAS-3486: introduce Namespace typedef
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71969/ --- (Updated Jan. 8, 2020, 1:23 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Fixing QuickStartV2It tests Repository: atlas Description --- ATLAS-3486: introduce Namespace typedef Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 3ebd70d981a734035416fdf9d880cebd68fe2f30 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 4742d1c0ea7d7a24fee0453d6fe052579ba6bbd2 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 3475ce6aa8dd4cad4caf08ab5501fc8dc37c1339 intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java b071dc9d664cee9e1ffc54726ffbf15f4f602d30 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java 7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 6016723d82abd47fa15c5f3a94b70c3ce0673413 repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 5bad615cd5c223b8246d8443ee5b33925a8b22e9 repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 08b00e76a5a96aa898f4fc252d3abc22845be50b repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java b04f188beff345d417141d194f0a61e584352111 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java fb56fad6412079e20dd3e345b81a08d9e5ace657 Diff: https://reviews.apache.org/r/71969/diff/2/ Changes: https://reviews.apache.org/r/71969/diff/1-2/ Testing --- As suggested by Madhan in this review here https://reviews.apache.org/r/71898/, it makes sense to have AtlasNamespaceDef inherit from AtlasStructDef. This change builds on top of that Thanks, Aadarsh Jajodia
Review Request 71969: ATLAS-3486: introduce Namespace typedef
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71969/ --- Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description --- ATLAS-3486: introduce Namespace typedef Diffs - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 3ebd70d981a734035416fdf9d880cebd68fe2f30 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 4742d1c0ea7d7a24fee0453d6fe052579ba6bbd2 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 3475ce6aa8dd4cad4caf08ab5501fc8dc37c1339 intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java b071dc9d664cee9e1ffc54726ffbf15f4f602d30 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java 7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 6016723d82abd47fa15c5f3a94b70c3ce0673413 repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 5bad615cd5c223b8246d8443ee5b33925a8b22e9 repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 08b00e76a5a96aa898f4fc252d3abc22845be50b repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java b04f188beff345d417141d194f0a61e584352111 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java fb56fad6412079e20dd3e345b81a08d9e5ace657 Diff: https://reviews.apache.org/r/71969/diff/1/ Testing --- As suggested by Madhan in this review here https://reviews.apache.org/r/71898/, it makes sense to have AtlasNamespaceDef inherit from AtlasStructDef. This change builds on top of that Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
> On Dec. 2, 2019, 5:06 a.m., Madhan Neethiraj wrote: > > intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java > > Lines 126 (patched) > > <https://reviews.apache.org/r/71649/diff/8/?file=2174818#file2174818line126> > > > > Having AtlasNamespaceAttribute extend from AtlasAttribute can be > > helpful in many cases, like: > > - write atrribute values in EntityGraphMapper > > - read attribute values in EntityGraphRetriever > > - SolrIndexHelper > > - perhaps in search as well (SearchProcessor) > > > > However, this would require: > > - AtlasNamespace extend from AtlasStructType (similr to > > AtlasClassificationType, AtlasEntityType) > > - AtlasNamespaceDef extend from AtlasStructDef > > > > Please review the design for above. Hi Madhan. Thanks for reviewing. The problem with AtlasNamespaceDef extending from AtlasStructDef, is that the attributes in a structDef are not tied to a specific entity type, like we want for the attributes in a Namespace Def. AtlasStructDef. AtlasStructDef has a list of List but those attributes are independent of any entity type, hence I was not able to extend AtlasNamespaceDef from AtlasStructDef. - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/#review218870 --- On Nov. 22, 2019, 2:03 a.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71649/ > --- > > (Updated Nov. 22, 2019, 2:03 a.m.) > > > Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3486 > https://issues.apache.org/jira/browse/ATLAS-3486 > > > Repository: atlas > > > Description > --- > > This change is the first part of the bigger task defined as part of > ATLAS-3485. > This adds the data model needed for supporting namespaces and namespace > attributes and also updates the type registry to include the applicable > namespace attributes for every entity type > > > Diffs > - > > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java > 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 > intg/src/main/java/org/apache/atlas/model/TypeCategory.java > f06f64f450f407e3f9a0e742726ff4dd12ccc695 > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de > intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java > 3634fdfd313639eb97b3c4698e091487b0e44a80 > intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java > 4ee68a936f99bb4c819b5335da2cc8bf7d539397 > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 > intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java > b071dc9d664cee9e1ffc54726ffbf15f4f602d30 > intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java > 0883d54f490e22c6510e6fc0cb804b87713a7ecb > intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java > dba2d88146eff314191ae6bb24ad7337b0ea10ae > intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java > 02613b5f7250b14324ed294c22de079b74d55b08 > intg/src/test/java/org/apache/atlas/TestUtilsV2.java > ff79994c519702e90b2e478d00cae0008889f956 > > intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java > 7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 > > repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java > 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java > 2cb2b47898ded4d6c5c84800ff93fa58b2c480da > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java > a5ccfb5b2055c88f596312f4033bc0034d3d165c > > repository/src/test/java/org/apache/atlas/repository/store/graph/v2/
Re: Review Request 71818: ATLAS-3534 Changes to allow namespaces to be added to an entity instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71818/ --- (Updated Nov. 26, 2019, 2:04 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Bugs: ATLAS-3534 https://issues.apache.org/jira/browse/ATLAS-3534 Repository: atlas Description (updated) --- The changes made in these include adding/updating namespaces to an entity instance as part of create or update Diffs - common/src/main/java/org/apache/atlas/repository/Constants.java 0b28243c47d9e025ecc38a05ce1a54efd71e6149 intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 1b033b9ac0d413c04269b480562940fb5b305939 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 1e7acf1e728f472a01a26fe4be69a11e21ca3e77 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityGraphDiscoveryV2.java d3d9367fb158f2bafbcc44f3c25ee1a63c5c2aa7 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java c8e65ef15fcf38cb4b5f1d9f1f37cf1578c13b48 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java a114d252e7462a08c14ebef598ed13b6b43e247b repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java 6b111732a11b2aef7dbeb050087bde3c70fd21bd Diff: https://reviews.apache.org/r/71818/diff/1/ Testing --- Thanks, Aadarsh Jajodia
Review Request 71818: ATLAS-3534 Changes to allow namespaces to be added to an entity instance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71818/ --- Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Bugs: ATLAS-3534 https://issues.apache.org/jira/browse/ATLAS-3534 Repository: atlas Description --- The changes made in these include adding/updating namespaces to an entity instance as part of create or update (cherry picked from commit d4bb3b8400a5cbb760aaa44aec7ea252d2f629f9) Diffs - common/src/main/java/org/apache/atlas/repository/Constants.java 0b28243c47d9e025ecc38a05ce1a54efd71e6149 intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 1b033b9ac0d413c04269b480562940fb5b305939 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 1e7acf1e728f472a01a26fe4be69a11e21ca3e77 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityGraphDiscoveryV2.java d3d9367fb158f2bafbcc44f3c25ee1a63c5c2aa7 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java c8e65ef15fcf38cb4b5f1d9f1f37cf1578c13b48 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java a114d252e7462a08c14ebef598ed13b6b43e247b repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java 6b111732a11b2aef7dbeb050087bde3c70fd21bd Diff: https://reviews.apache.org/r/71818/diff/1/ Testing --- Thanks, Aadarsh Jajodia
[jira] [Created] (ATLAS-3534) EntityREST changes to allow namespace attributes to be added to an entity instance
Aadarsh Jajodia created ATLAS-3534: -- Summary: EntityREST changes to allow namespace attributes to be added to an entity instance Key: ATLAS-3534 URL: https://issues.apache.org/jira/browse/ATLAS-3534 Project: Atlas Issue Type: New Feature Reporter: Aadarsh Jajodia Assignee: Aadarsh Jajodia -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Nov. 22, 2019, 2:03 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Addressed Review comments Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes and also updates the type registry to include the applicable namespace attributes for every entity type Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java b071dc9d664cee9e1ffc54726ffbf15f4f602d30 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/TestUtilsV2.java ff79994c519702e90b2e478d00cae0008889f956 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java 7c551304b2b65b90302f6e5fa9fc5b9f1b8e2c12 repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasAbstractDefStoreV2.java 2cb2b47898ded4d6c5c84800ff93fa58b2c480da repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a webapp/src/main/java/org/apache/atlas/web/rest/TypesREST.java fb56fad6412079e20dd3e345b81a08d9e5ace657 Diff: https://reviews.apache.org/r/71649/diff/8/ Changes: https://reviews.apache.org/r/71649/diff/7-8/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Nov. 21, 2019, 3:03 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Rebased Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes and also updates the type registry to include the applicable namespace attributes for every entity type Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java bb7ead0f9f8bab3094eb82e9e286dd58e8a6e3de intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 557ef74a95c2a939b4b89cd1db8fa4c73d52dd51 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java b071dc9d664cee9e1ffc54726ffbf15f4f602d30 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/TestUtilsV2.java ff79994c519702e90b2e478d00cae0008889f956 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java 51dd16b8518c9a16088547f3e95c0ef401695895 repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a Diff: https://reviews.apache.org/r/71649/diff/7/ Changes: https://reviews.apache.org/r/71649/diff/6-7/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Nov. 20, 2019, 2:24 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Updated patch to handle cases when applicable entity types cannot be removed as part of a namespaceDef update call Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes and also updates the type registry to include the applicable namespace attributes for every entity type Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 884447f81c57faf917a8d0565fc0a0c7ebbd99f0 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 8b4fd1c3b9005d0a8852f2828475b4ad6a806822 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/TestUtilsV2.java 530d5cda44b7d3746f5cb8dd5a23b3c68b254cac intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java 51dd16b8518c9a16088547f3e95c0ef401695895 repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a Diff: https://reviews.apache.org/r/71649/diff/6/ Changes: https://reviews.apache.org/r/71649/diff/5-6/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Nov. 19, 2019, 11:07 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Addressed review comments. Also added unit tests to validate applicable entity types Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes and also updates the type registry to include the applicable namespace attributes for every entity type Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 884447f81c57faf917a8d0565fc0a0c7ebbd99f0 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 8b4fd1c3b9005d0a8852f2828475b4ad6a806822 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/TestUtilsV2.java 530d5cda44b7d3746f5cb8dd5a23b3c68b254cac intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java 51dd16b8518c9a16088547f3e95c0ef401695895 repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a Diff: https://reviews.apache.org/r/71649/diff/5/ Changes: https://reviews.apache.org/r/71649/diff/4-5/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
> On Nov. 15, 2019, 1:03 a.m., Sarath Subramanian wrote: > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java > > Lines 264 (patched) > > <https://reviews.apache.org/r/71649/diff/4/?file=2173254#file2173254line264> > > > > this is not valid. NamespaceDef should be deleted only when no entity > > has reference to that namespace name. > > > > Maybe check if there are any entity referring this namespace (using > > index query?) before deleting. > > > > Same for line #294 So i can remove this check from this patch and when i submit the patch for the entity changes I can make this change for the delete method? - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/#review218640 ----------- On Nov. 14, 2019, 12:08 a.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71649/ > --- > > (Updated Nov. 14, 2019, 12:08 a.m.) > > > Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3486 > https://issues.apache.org/jira/browse/ATLAS-3486 > > > Repository: atlas > > > Description > --- > > This change is the first part of the bigger task defined as part of > ATLAS-3485. > This adds the data model needed for supporting namespaces and namespace > attributes and also updates the type registry to include the applicable > namespace attributes for every entity type > > > Diffs > - > > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java > 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 > intg/src/main/java/org/apache/atlas/model/TypeCategory.java > f06f64f450f407e3f9a0e742726ff4dd12ccc695 > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > e10965b870c4a300b41e93ee046b5f6d6b722728 > intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java > 3634fdfd313639eb97b3c4698e091487b0e44a80 > intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java > 4ee68a936f99bb4c819b5335da2cc8bf7d539397 > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 884447f81c57faf917a8d0565fc0a0c7ebbd99f0 > intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java > 8b4fd1c3b9005d0a8852f2828475b4ad6a806822 > intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java > 0883d54f490e22c6510e6fc0cb804b87713a7ecb > intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java > dba2d88146eff314191ae6bb24ad7337b0ea10ae > intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java > 02613b5f7250b14324ed294c22de079b74d55b08 > intg/src/test/java/org/apache/atlas/TestUtilsV2.java > 530d5cda44b7d3746f5cb8dd5a23b3c68b254cac > > intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java > 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java > a5ccfb5b2055c88f596312f4033bc0034d3d165c > > repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java > 51dd16b8518c9a16088547f3e95c0ef401695895 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java > 6cd0ee331b7ae24757b58e76ec47bf556106846a > > > Diff: https://reviews.apache.org/r/71649/diff/4/ > > > Testing > --- > > Added unit tests > > > Thanks, > > Aadarsh Jajodia > >
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Nov. 14, 2019, 12:08 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Addressed review comments Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes and also updates the type registry to include the applicable namespace attributes for every entity type Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2e9ae8174c5309164f3e41c940cbf3ddf8 intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 884447f81c57faf917a8d0565fc0a0c7ebbd99f0 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 8b4fd1c3b9005d0a8852f2828475b4ad6a806822 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/TestUtilsV2.java 530d5cda44b7d3746f5cb8dd5a23b3c68b254cac intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java 51dd16b8518c9a16088547f3e95c0ef401695895 repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a Diff: https://reviews.apache.org/r/71649/diff/4/ Changes: https://reviews.apache.org/r/71649/diff/3-4/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Nov. 8, 2019, 9:42 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Summary (updated) - ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas and also the corresponding type registry changes for the same Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description (updated) --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes and also updates the type registry to include the applicable namespace attributes for every entity type Diffs - intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/main/java/org/apache/atlas/model/typedef/AtlasTypesDef.java 3634fdfd313639eb97b3c4698e091487b0e44a80 intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java 4ee68a936f99bb4c819b5335da2cc8bf7d539397 intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 884447f81c57faf917a8d0565fc0a0c7ebbd99f0 intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 8b4fd1c3b9005d0a8852f2828475b4ad6a806822 intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 0883d54f490e22c6510e6fc0cb804b87713a7ecb intg/src/main/java/org/apache/atlas/typesystem/types/DataTypes.java dba2d88146eff314191ae6bb24ad7337b0ea10ae intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 02613b5f7250b14324ed294c22de079b74d55b08 intg/src/test/java/org/apache/atlas/TestUtilsV2.java 530d5cda44b7d3746f5cb8dd5a23b3c68b254cac intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java 2e2ab1a664171555c57560e1c0b4cbdbc20c0f6f repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasTypeDefGraphStoreV2.java a5ccfb5b2055c88f596312f4033bc0034d3d165c repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java 51dd16b8518c9a16088547f3e95c0ef401695895 repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasNamespaceDefStoreV2Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java 6f9c05e7a7a30c678a0a376d25402c6026d0b391 tools/classification-updater/src/main/java/org/apache/atlas/tools/BulkFetchAndUpdate.java 91ff89a00d779af9437ba5f6d129bd4595e5f036 webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6cd0ee331b7ae24757b58e76ec47bf556106846a Diff: https://reviews.apache.org/r/71649/diff/3/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
[jira] [Updated] (ATLAS-3501) Update guava version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3501: --- Attachment: 0001-ATLAS-3501-Update-guava-version-to-28.0-jre.patch > Update guava version to 28.0-jre > > > Key: ATLAS-3501 > URL: https://issues.apache.org/jira/browse/ATLAS-3501 > Project: Atlas > Issue Type: Bug > Reporter: Aadarsh Jajodia > Assignee: Aadarsh Jajodia >Priority: Major > Attachments: 0001-ATLAS-3501-Update-guava-version-to-28.0-jre.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3501) Update guava version to 28.0-jre
Aadarsh Jajodia created ATLAS-3501: -- Summary: Update guava version to 28.0-jre Key: ATLAS-3501 URL: https://issues.apache.org/jira/browse/ATLAS-3501 Project: Atlas Issue Type: Bug Reporter: Aadarsh Jajodia Assignee: Aadarsh Jajodia -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Summary: Update jackson.databind.version to 2.10.0 (was: Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre) > Update jackson.databind.version to 2.10.0 > - > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Kalman >Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk, 2.1.0 > > Attachments: > 0001-ATLAS-3452-Update-jackson.databind.version-to-2.10.0.patch > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Attachment: 0001-ATLAS-3452-Update-jackson.databind.version-to-2.10.0.patch > Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre > --- > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Kalman >Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk, 2.1.0 > > Attachments: > 0001-ATLAS-3452-Update-jackson.databind.version-to-2.10.0.patch > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Attachment: (was: 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version to 28.0-jre.patch) > Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre > --- > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Kalman >Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk, 2.1.0 > > Attachments: > 0001-ATLAS-3452-Update-jackson.databind.version-to-2.10.0.patch > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Attachment: 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version to 28.0-jre.patch > Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre > --- > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Kalman >Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk, 2.1.0 > > Attachments: > 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version > to 28.0-jre.patch > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Attachment: (was: 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version to 28.0-jre.patch) > Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre > --- > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Kalman >Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk, 2.1.0 > > Attachments: > 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version > to 28.0-jre.patch > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Attachment: 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version to 28.0-jre.patch > Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre > --- > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Kalman > Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk > > Attachments: > 0001-ATLAS-3452-Updating-jackson.databind.version-to-2.10 and Guava Version > to 28.0-jre.patch > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3452) Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3452: --- Summary: Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre (was: Update jackson.databind.version to 2.10.0) > Update jackson.databind.version to 2.10.0 and Guava Version to 28.0-jre > --- > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Kalman > Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-3452) Update jackson.databind.version to 2.10.0
[ https://issues.apache.org/jira/browse/ATLAS-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia reassigned ATLAS-3452: -- Assignee: Aadarsh Jajodia > Update jackson.databind.version to 2.10.0 > - > > Key: ATLAS-3452 > URL: https://issues.apache.org/jira/browse/ATLAS-3452 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Kalman > Assignee: Aadarsh Jajodia >Priority: Major > Fix For: trunk > > > Atlas uses jackson-databind 2.9.9.1, but there is a patch release, 2.10.0 to > address a CVE vulnerability, to make static code analysis tool happy, we need > to upgrade it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 71681: ATLAS-3494 Fixing build failure in branch-2.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71681/ --- Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description --- ATLAS-3494 Fixing build failure in branch-2.0 Diffs - tools/atlas-index-repair/pom.xml 7330e12ce89c580f173626327b2993aa036a6c92 tools/classification-updater/pom.xml ea7250c541d131957d3a573012cd5dae5600a568 Diff: https://reviews.apache.org/r/71681/diff/1/ Testing --- Thanks, Aadarsh Jajodia
Re: Review Request 71681: ATLAS-3494 Fixing build failure in branch-2.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71681/ --- (Updated Oct. 25, 2019, 6:33 p.m.) Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description --- ATLAS-3494 Fixing build failure in branch-2.0 Diffs - tools/atlas-index-repair/pom.xml 7330e12ce89c580f173626327b2993aa036a6c92 tools/classification-updater/pom.xml ea7250c541d131957d3a573012cd5dae5600a568 Diff: https://reviews.apache.org/r/71681/diff/1/ Testing (updated) --- Verified build is working fine. Ran mvn clean install -DskipTests Thanks, Aadarsh Jajodia
Re: Review Request 71681: ATLAS-3494 Fixing build failure in branch-2.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71681/ --- (Updated Oct. 25, 2019, 6:33 p.m.) Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian. Bugs: ATLAS-3494 https://issues.apache.org/jira/browse/ATLAS-3494 Repository: atlas Description --- ATLAS-3494 Fixing build failure in branch-2.0 Diffs - tools/atlas-index-repair/pom.xml 7330e12ce89c580f173626327b2993aa036a6c92 tools/classification-updater/pom.xml ea7250c541d131957d3a573012cd5dae5600a568 Diff: https://reviews.apache.org/r/71681/diff/1/ Testing --- Verified build is working fine. Ran mvn clean install -DskipTests Thanks, Aadarsh Jajodia
[jira] [Updated] (ATLAS-3494) Build failure in branch-2.0
[ https://issues.apache.org/jira/browse/ATLAS-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3494: --- Attachment: 0001-ATLAS-3494-Fixing-build-failure-in-branch-2.0.patch > Build failure in branch-2.0 > --- > > Key: ATLAS-3494 > URL: https://issues.apache.org/jira/browse/ATLAS-3494 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Aadarsh Jajodia > Assignee: Aadarsh Jajodia >Priority: Major > Attachments: 0001-ATLAS-3494-Fixing-build-failure-in-branch-2.0.patch > > > Incorrect version number in these pom files > * tools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml > * tools/classification-updater/pom.xml > is causing the build to fail on branch-2.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-3494) Build failure in branch-2.0
[ https://issues.apache.org/jira/browse/ATLAS-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia reassigned ATLAS-3494: -- Assignee: Aadarsh Jajodia > Build failure in branch-2.0 > --- > > Key: ATLAS-3494 > URL: https://issues.apache.org/jira/browse/ATLAS-3494 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Aadarsh Jajodia > Assignee: Aadarsh Jajodia >Priority: Major > > Incorrect version number in thse pom file > fortools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3494) Build failure in branch-2.0
[ https://issues.apache.org/jira/browse/ATLAS-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3494: --- Description: Incorrect version number in these pom files * tools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml * tools/classification-updater/pom.xml is causing the build to fail on branch-2.0 was:Incorrect version number in thse pom file fortools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml > Build failure in branch-2.0 > --- > > Key: ATLAS-3494 > URL: https://issues.apache.org/jira/browse/ATLAS-3494 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Aadarsh Jajodia > Assignee: Aadarsh Jajodia >Priority: Major > > Incorrect version number in these pom files > * tools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml > * tools/classification-updater/pom.xml > is causing the build to fail on branch-2.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3494) Build failure in branch-2.0
[ https://issues.apache.org/jira/browse/ATLAS-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3494: --- Description: Incorrect version number in thse pom file fortools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml > Build failure in branch-2.0 > --- > > Key: ATLAS-3494 > URL: https://issues.apache.org/jira/browse/ATLAS-3494 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Aadarsh Jajodia >Priority: Major > > Incorrect version number in thse pom file > fortools/atlas-index-repair/pom.xml, tools/classification-updater/pom.xml -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3494) Build failure in branch-2.0
Aadarsh Jajodia created ATLAS-3494: -- Summary: Build failure in branch-2.0 Key: ATLAS-3494 URL: https://issues.apache.org/jira/browse/ATLAS-3494 Project: Atlas Issue Type: Bug Affects Versions: 2.1.0 Reporter: Aadarsh Jajodia -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
> On Oct. 23, 2019, 8:20 p.m., Sarath Subramanian wrote: > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > > Lines 155 (patched) > > <https://reviews.apache.org/r/71649/diff/1/?file=2169832#file2169832line155> > > > > displayName is nto used; consider removing it. Setting the displayName in the constructor for AtlasNamespaceAttributeDef - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/#review218367 --- On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71649/ > --- > > (Updated Oct. 22, 2019, 9:08 p.m.) > > > Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3486 > https://issues.apache.org/jira/browse/ATLAS-3486 > > > Repository: atlas > > > Description > --- > > This change is the first part of the bigger task defined as part of > ATLAS-3485. > This adds the data model needed for supporting namespaces and namespace > attributes > > > Diffs > - > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > e10965b870c4a300b41e93ee046b5f6d6b722728 > > intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/71649/diff/1/ > > > Testing > --- > > Added unit tests > > > Thanks, > > Aadarsh Jajodia > >
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Oct. 24, 2019, 11:32 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Addressed Review Comments Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes Diffs (updated) - intg/src/main/java/org/apache/atlas/model/TypeCategory.java f06f64f450f407e3f9a0e742726ff4dd12ccc695 intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION Diff: https://reviews.apache.org/r/71649/diff/2/ Changes: https://reviews.apache.org/r/71649/diff/1-2/ Testing --- Added unit tests Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
> On Oct. 22, 2019, 9:38 p.m., Le Ma wrote: > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > > Lines 106 (patched) > > <https://reviews.apache.org/r/71649/diff/1/?file=2169832#file2169832line106> > > > > instead of using a variable i, Why not using for (int i = 0; i < > > xx.size(); i++), it would be cleaner and more intitive, only when i is the > > laste index, you can stop appending "," so since attributeDefs is a Set, cannot iterate over a set using this kind of construct for (int i = 0; i < xx.size(); i++). We will have to use an iterator for that. Hence I am dropping this comment. > On Oct. 22, 2019, 9:38 p.m., Le Ma wrote: > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > > Lines 207 (patched) > > <https://reviews.apache.org/r/71649/diff/1/?file=2169832#file2169832line207> > > > > Consider using the result of toString() to get hashcode. updated the hashcode to include all the attibutes of the object > On Oct. 22, 2019, 9:38 p.m., Le Ma wrote: > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > > Lines 211 (patched) > > <https://reviews.apache.org/r/71649/diff/1/?file=2169832#file2169832line211> > > > > Why not merging two toString() functions? create the stringbuilder > > inside, and return sb.toString() ? So we want to have both methods because the class AtlasBaseTypeDef only overrides toString(StringBuilder) - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/#review218343 --- On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71649/ > --- > > (Updated Oct. 22, 2019, 9:08 p.m.) > > > Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3486 > https://issues.apache.org/jira/browse/ATLAS-3486 > > > Repository: atlas > > > Description > --- > > This change is the first part of the bigger task defined as part of > ATLAS-3485. > This adds the data model needed for supporting namespaces and namespace > attributes > > > Diffs > - > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > e10965b870c4a300b41e93ee046b5f6d6b722728 > > intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/71649/diff/1/ > > > Testing > --- > > Added unit tests > > > Thanks, > > Aadarsh Jajodia > >
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
> On Oct. 23, 2019, 12:30 a.m., Madhan Neethiraj wrote: > > - please update AtlasTypesDef to add attribute: private > > List namespaceDefs; Will be adding this in the next update of this patch - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/#review218345 --- On Oct. 22, 2019, 9:08 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71649/ > --- > > (Updated Oct. 22, 2019, 9:08 p.m.) > > > Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3486 > https://issues.apache.org/jira/browse/ATLAS-3486 > > > Repository: atlas > > > Description > --- > > This change is the first part of the bigger task defined as part of > ATLAS-3485. > This adds the data model needed for supporting namespaces and namespace > attributes > > > Diffs > - > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > e10965b870c4a300b41e93ee046b5f6d6b722728 > > intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/71649/diff/1/ > > > Testing > --- > > Added unit tests > > > Thanks, > > Aadarsh Jajodia > >
Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes Testing done: Added unit tests Diffs - intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION Diff: https://reviews.apache.org/r/71649/diff/1/ Testing --- Thanks, Aadarsh Jajodia
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Oct. 22, 2019, 9:08 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description (updated) --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes Diffs - intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION Diff: https://reviews.apache.org/r/71649/diff/1/ Testing (updated) --- Added unit tests Thanks, Aadarsh Jajodia
[jira] [Updated] (ATLAS-3486) Define Data Models For Namespaces and Namespace Attributes in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3486: --- Description: This change defines the data models needed for namespace definition and namespace attribute definitions > Define Data Models For Namespaces and Namespace Attributes in Atlas > --- > > Key: ATLAS-3486 > URL: https://issues.apache.org/jira/browse/ATLAS-3486 > Project: Atlas > Issue Type: Sub-task > Reporter: Aadarsh Jajodia > Assignee: Aadarsh Jajodia >Priority: Major > > This change defines the data models needed for namespace definition and > namespace attribute definitions -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71649: ATLAS-3486 Define Data Models For Namespaces and Namespace Attributes in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71649/ --- (Updated Oct. 22, 2019, 8:58 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Bugs: ATLAS-3486 https://issues.apache.org/jira/browse/ATLAS-3486 Repository: atlas Description --- This change is the first part of the bigger task defined as part of ATLAS-3485. This adds the data model needed for supporting namespaces and namespace attributes Testing done: Added unit tests Diffs - intg/src/main/java/org/apache/atlas/model/typedef/AtlasNamespaceDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java e10965b870c4a300b41e93ee046b5f6d6b722728 intg/src/test/java/org/apache/atlas/model/typedef/TestAtlasNamespaceDef.java PRE-CREATION Diff: https://reviews.apache.org/r/71649/diff/1/ Testing --- Thanks, Aadarsh Jajodia
[jira] [Created] (ATLAS-3486) Define Data Models For Namespaces and Namespace Attributes in Atlas
Aadarsh Jajodia created ATLAS-3486: -- Summary: Define Data Models For Namespaces and Namespace Attributes in Atlas Key: ATLAS-3486 URL: https://issues.apache.org/jira/browse/ATLAS-3486 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia Assignee: Aadarsh Jajodia -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3485) Introducing attribute namespaces In Atlas(Managed Metadata)
Aadarsh Jajodia created ATLAS-3485: -- Summary: Introducing attribute namespaces In Atlas(Managed Metadata) Key: ATLAS-3485 URL: https://issues.apache.org/jira/browse/ATLAS-3485 Project: Atlas Issue Type: New Feature Reporter: Aadarsh Jajodia Assignee: Aadarsh Jajodia As a data steward I would like to be able to add additional custom attributes to existing types and set their values on existing entities for the following use case: I want to be able to add organizationally relevant 'sanctioned and curated' master data: extend my enterprise data model with attributes that have specific meaning for my business and a domain of values that I can define and enforce. For e.g. I can augment technical metadata with business level master data such as setting my lines of business with an enumerated list of values such as Commercial, Personal Banking, Retail banking, Small Business Banking, Global Wealth management. Implementation considerations: * Freeform key-value pairs * Managed data attributes with specific domain of values and enforcement. * Be able to associate additional attributes (name/value pairs) with existing entities * Both UI based addition/management of properties but also an API bridge or scripts/utility for setting/updating such custom properties in bulk (for bootstrapping or periodic updates) across multiple entities need to be supported -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71108: ATLAS-3287: Adding functionality for generating and setting dynamic attributes for entities with all references within the entity stream
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71108/#review217097 --- intg/src/main/java/org/apache/atlas/utils/EntityFinder.java Lines 25 (patched) <https://reviews.apache.org/r/71108/#comment304356> Should this be renamed to getEntityFromEntityStream? intg/src/main/java/org/apache/atlas/utils/EntityFinder.java Lines 27 (patched) <https://reviews.apache.org/r/71108/#comment304357> Should this be renamed to getEntityFromEntityVertex? - Aadarsh Jajodia On July 30, 2019, 11:54 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71108/ > --- > > (Updated July 30, 2019, 11:54 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > Dynamic Attributes for the current entity are generated and set. > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 23eaa0a2e88fd348d2347314170726ebb5cb4393 > intg/src/main/java/org/apache/atlas/type/AttributeToken.java > 658cf865ec1fe4eb326930024c4f189f59df0e7e > intg/src/main/java/org/apache/atlas/type/ConstantToken.java > 5ba54ae6333440b633e3b6a1175b384676004cf9 > intg/src/main/java/org/apache/atlas/type/DependentToken.java > c1c7d3df70118dd127b74d4ece65a47185258bc4 > intg/src/main/java/org/apache/atlas/type/TemplateToken.java > fffbc2ae3ff72144a2779437c600f42d684cf548 > intg/src/main/java/org/apache/atlas/utils/EntityFinder.java PRE-CREATION > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > c114bdf7888519cdd396fc8501bbcd92b00ef0bb > > repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java > 01b8d196b354cf089fe4b040e396761c9a19a7eb > > repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java > 2221ac4f421fbdbb2d8d39889863223568a3ae70 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityFinder.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityGraphDiscoveryV2.java > 84bb2a363358d213611d321c01cb5f14078ae09d > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java > 983d4fa7250aec40f0b83d586a5a71a063bbed62 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStream.java > c823b20079ab9ff61fc0ff9b59ccb6b301c0664d > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStreamForImport.java > 6bf962e8b206b0bc320f3268e269e9292de08b84 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityImportStream.java > cf7ac28f9d6f0d86a2c92a984a331e818a4f5f9b > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityStream.java > 41b834204b077e7a866198fead736298524787fa > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/IDBasedEntityResolver.java > fe76b3a8943180f8a4332b6363ddc782da321aa4 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/InMemoryMapEntityStream.java > e1ade0709779bfd9acc5532434bdab12c6236591 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2Test.java > a4edaf086ebc162cf6343e9ea439185f495691e6 > webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java > e5b9563d6005372e8acd11633542af59bca1ef78 > > webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java > 940c639cba02763887843ed14b8240bfc7569631 > > webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java > ece46a42b77d5fb5db2b9c1b042cc5327747c1e3 > > > Diff: https://reviews.apache.org/r/71108/diff/3/ > > > Testing > --- > > Tested with creates and updates, dynamicAttributes are calculated correctly. > Ran mvn install for both the intg and repository packages. > > > Thanks, > > Merryle Wang > >
Re: Review Request 71108: ATLAS-3287: Adding functionality for generating and setting dynamic attributes for entities with all references within the entity stream
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71108/#review217096 --- intg/src/main/java/org/apache/atlas/type/DependentToken.java Lines 51-72 (patched) <https://reviews.apache.org/r/71108/#comment304355> Instead of having these null checks can we use Optional type in Java? We can avoid making those null checks. repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityFinder.java Lines 30 (patched) <https://reviews.apache.org/r/71108/#comment304354> Can we add more comments in this class for every method to understand the sequence of which method is called for which scenario? Like for example the getByGuid add a comment saying trying to fetch the object from the entity stream and so on for the other methods. Also can we rename getByGuid to getFromEntityStream? since actually we are just getting the object from the entityStream in that method. repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java Lines 887-893 (patched) <https://reviews.apache.org/r/71108/#comment304353> Do we need to do this everytime? We should be only doing this for an update which involves dynamicAttribute or a dependent one. - Aadarsh Jajodia On July 30, 2019, 11:54 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71108/ > --- > > (Updated July 30, 2019, 11:54 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > Dynamic Attributes for the current entity are generated and set. > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 23eaa0a2e88fd348d2347314170726ebb5cb4393 > intg/src/main/java/org/apache/atlas/type/AttributeToken.java > 658cf865ec1fe4eb326930024c4f189f59df0e7e > intg/src/main/java/org/apache/atlas/type/ConstantToken.java > 5ba54ae6333440b633e3b6a1175b384676004cf9 > intg/src/main/java/org/apache/atlas/type/DependentToken.java > c1c7d3df70118dd127b74d4ece65a47185258bc4 > intg/src/main/java/org/apache/atlas/type/TemplateToken.java > fffbc2ae3ff72144a2779437c600f42d684cf548 > intg/src/main/java/org/apache/atlas/utils/EntityFinder.java PRE-CREATION > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > c114bdf7888519cdd396fc8501bbcd92b00ef0bb > > repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java > 01b8d196b354cf089fe4b040e396761c9a19a7eb > > repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java > 2221ac4f421fbdbb2d8d39889863223568a3ae70 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityFinder.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityGraphDiscoveryV2.java > 84bb2a363358d213611d321c01cb5f14078ae09d > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java > 983d4fa7250aec40f0b83d586a5a71a063bbed62 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStream.java > c823b20079ab9ff61fc0ff9b59ccb6b301c0664d > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStreamForImport.java > 6bf962e8b206b0bc320f3268e269e9292de08b84 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityImportStream.java > cf7ac28f9d6f0d86a2c92a984a331e818a4f5f9b > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityStream.java > 41b834204b077e7a866198fead736298524787fa > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/IDBasedEntityResolver.java > fe76b3a8943180f8a4332b6363ddc782da321aa4 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/InMemoryMapEntityStream.java > e1ade0709779bfd9acc5532434bdab12c6236591 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2Test.java > a4edaf086ebc162cf6343e9ea439185f495691e6 > webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java > e5b9563d6005372e8acd11633542af59bca1ef78 > > webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java > 940c639cba02763887843ed14b8240bfc7569631 > > webapp/src/test/java/o
Re: Review Request 71127: ATLAS-3343: Ordering of dynAttr evaluation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71127/#review216871 --- Ship it! LGTM!! - Aadarsh Jajodia On July 25, 2019, 8:20 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71127/ > --- > > (Updated July 25, 2019, 8:20 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3343: Ordering of dynAttr evaluation > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 23eaa0a2e88fd348d2347314170726ebb5cb4393 > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > c114bdf7888519cdd396fc8501bbcd92b00ef0bb > > > Diff: https://reviews.apache.org/r/71127/diff/3/ > > > Testing > --- > > Created dummy dynamic attributes to test the correct ordering. > > > Thanks, > > Merryle Wang > >
Re: Review Request 71127: ATLAS-3343: Ordering of dynAttr evaluation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71127/#review216865 --- intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java Lines 1015 (patched) <https://reviews.apache.org/r/71127/#comment304115> If you are using a stack here then you don't need to reverse the order. Also if you are using a stack and you are also reversing the order then the test case should have failed. Can you check if the test is working properly? - Aadarsh Jajodia On July 23, 2019, 11:22 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71127/ > --- > > (Updated July 23, 2019, 11:22 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3343: Ordering of dynAttr evaluation > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 23eaa0a2e88fd348d2347314170726ebb5cb4393 > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > c114bdf7888519cdd396fc8501bbcd92b00ef0bb > > > Diff: https://reviews.apache.org/r/71127/diff/2/ > > > Testing > --- > > Created dummy dynamic attributes to test the correct ordering. > > > Thanks, > > Merryle Wang > >
Re: Review Request 71127: ATLAS-3343: Ordering of dynAttr evaluation
> On July 23, 2019, 8:17 p.m., Aadarsh Jajodia wrote:
> > Can we use a simple adjacency list to represent the graph. Something like a
> > Map > to represent the graph. And also I feel we can
> > just use a visited set and stack to maintain the topological order? Maybe
> > that's simpler?
Also for the toplogocial sort using the stack, you could follow this approach.
void topologicalSort(string vertex) {
visited.add(vertex);
for (neighbor in vertex) {
if(!visited[neighbor]) topologicalSort(neighbor)
}
stack.push(vertex);
}
We can pop all the elements in the stack and that should give the answer.
- Aadarsh
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71127/#review216812
---
On July 19, 2019, 6:49 p.m., Merryle Wang wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71127/
> ---
>
> (Updated July 19, 2019, 6:49 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma,
> Madhan Neethiraj, and Sarath Subramanian.
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-3343: Ordering of dynAttr evaluation
>
>
> Diffs
> -
>
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
> 23eaa0a2e88fd348d2347314170726ebb5cb4393
>
>
> Diff: https://reviews.apache.org/r/71127/diff/1/
>
>
> Testing
> ---
>
> Created dummy dynamic attributes to test the correct ordering.
>
>
> Thanks,
>
> Merryle Wang
>
>
Re: Review Request 71127: ATLAS-3343: Ordering of dynAttr evaluation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71127/#review216812 --- Can we use a simple adjacency list to represent the graph. Something like a Map > to represent the graph. And also I feel we can just use a visited set and stack to maintain the topological order? Maybe that's simpler? - Aadarsh Jajodia On July 19, 2019, 6:49 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71127/ > --- > > (Updated July 19, 2019, 6:49 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3343: Ordering of dynAttr evaluation > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > 23eaa0a2e88fd348d2347314170726ebb5cb4393 > > > Diff: https://reviews.apache.org/r/71127/diff/1/ > > > Testing > --- > > Created dummy dynamic attributes to test the correct ordering. > > > Thanks, > > Merryle Wang > >
Re: Review Request 70913: ATLAS-3286: Populated dynamic attribute flags for AtlasAttributes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70913/#review216632 --- Ship it! Thanks for uploading the model file. We can have that as a reference. - Aadarsh Jajodia On July 16, 2019, 12:12 a.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70913/ > --- > > (Updated July 16, 2019, 12:12 a.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3286: Populated dynamic attribute flags for AtlasAttributes > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > d9ae9e36773146fad652a1e28fc8822bae5c8557 > intg/src/main/java/org/apache/atlas/type/AtlasStructType.java > 0fe47bdcbe84c06545c517fec3177ef7e6487a6c > intg/src/main/java/org/apache/atlas/type/AttributeToken.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/ConstantToken.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/DependentToken.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/TemplateToken.java PRE-CREATION > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > 3c53c02b48747515217c9327c98209a48ee84237 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityGraphDiscoveryV2.java > 4ff4206471bb8b2b4997c00dca3b4433626ae392 > > webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java > bf7e182429d00ca668cbbd41277291ecc5d457e0 > > webapp/src/test/java/org/apache/atlas/web/integration/EntityJerseyResourceIT.java > edb45687712c7bb8941bc894e73423a463739a42 > > > Diff: https://reviews.apache.org/r/70913/diff/14/ > > > Testing > --- > > Added unit test > > > File Attachments > > > Hive Model with dynAttributes > > https://reviews.apache.org/media/uploaded/files/2019/07/16/1f7bed33-0509-4aee-acbf-7ed1b0616c84__1030-hive_model.json > > > Thanks, > > Merryle Wang > >
Re: Review Request 70913: ATLAS-3286: Populated dynamic attribute flags for AtlasAttributes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70913/#review216630 --- Ship it! Ship It! - Aadarsh Jajodia On July 15, 2019, 11:09 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70913/ > --- > > (Updated July 15, 2019, 11:09 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3286: Populated dynamic attribute flags for AtlasAttributes > > > Diffs > - > > addons/models/1000-Hadoop/1030-hive_model.json > 8901aa4aa86fb2802a9e9b1e65c2ff8aad8855ad > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > d9ae9e36773146fad652a1e28fc8822bae5c8557 > intg/src/main/java/org/apache/atlas/type/AtlasStructType.java > 0fe47bdcbe84c06545c517fec3177ef7e6487a6c > intg/src/main/java/org/apache/atlas/type/AttributeToken.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/ConstantToken.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/DependentToken.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/TemplateToken.java PRE-CREATION > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > 3c53c02b48747515217c9327c98209a48ee84237 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityGraphDiscoveryV2.java > 4ff4206471bb8b2b4997c00dca3b4433626ae392 > > webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java > bf7e182429d00ca668cbbd41277291ecc5d457e0 > > webapp/src/test/java/org/apache/atlas/web/integration/EntityJerseyResourceIT.java > edb45687712c7bb8941bc894e73423a463739a42 > > > Diff: https://reviews.apache.org/r/70913/diff/13/ > > > Testing > --- > > Added unit test > > > Thanks, > > Merryle Wang > >
[jira] [Updated] (ATLAS-3336) Propagating Dynamic Attribute changes
[
https://issues.apache.org/jira/browse/ATLAS-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aadarsh Jajodia updated ATLAS-3336:
---
Description:
attr1
attr2 - \{attr1}_name
attr3 : \{attr2}_value
Here attr2 and attr3 are dynamic attributes
The order in which these should be computed is
# attr2
# attr3
The reason is attr3 is dependent on attr2
The same propagation logic should hold when we change a database name
table - \{db.name}..
column - \{table.db.name}.\{table.name}
Here table's qualifiedName should be updated first and then the column's name
was:
attr1
attr2 - \{attr1}_name
attr3 : \{attr2}_value
Here attr2 and attr3 are dynamic attributes
The order in which these should be computed is that first attr2 and then attr3.
The same propagation logic should hold when we change a database name
table - \{db.name}..
column - \{table.db.name}.\{table.name}
Here table's qualifiedName should be updated first and then the column's name
> Propagating Dynamic Attribute changes
> -
>
> Key: ATLAS-3336
> URL: https://issues.apache.org/jira/browse/ATLAS-3336
> Project: Atlas
> Issue Type: Bug
> Reporter: Aadarsh Jajodia
>Assignee: Merryle Wang
>Priority: Major
>
> attr1
> attr2 - \{attr1}_name
> attr3 : \{attr2}_value
>
> Here attr2 and attr3 are dynamic attributes
> The order in which these should be computed is
> # attr2
> # attr3
> The reason is attr3 is dependent on attr2
>
> The same propagation logic should hold when we change a database name
> table - \{db.name}..
> column - \{table.db.name}.\{table.name}
>
> Here table's qualifiedName should be updated first and then the column's name
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (ATLAS-3336) Propagating Dynamic Attribute changes
Aadarsh Jajodia created ATLAS-3336:
--
Summary: Propagating Dynamic Attribute changes
Key: ATLAS-3336
URL: https://issues.apache.org/jira/browse/ATLAS-3336
Project: Atlas
Issue Type: Bug
Reporter: Aadarsh Jajodia
Assignee: Merryle Wang
attr1
attr2 - \{attr1}_name
attr3 : \{attr2}_value
Here attr2 and attr3 are dynamic attributes
The order in which these should be computed is that first attr2 and then attr3.
The same propagation logic should hold when we change a database name
table - \{db.name}..
column - \{table.db.name}.\{table.name}
Here table's qualifiedName should be updated first and then the column's name
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
Re: Review Request 70913: ATLAS-3286: Populated dynamic attribute flags for AtlasAttributes
> On July 2, 2019, 3:50 p.m., Ashutosh Mestry wrote: > > This an edge case: When temporary tables are created, their qualifiedName > > gets suffixed with GUID, how will this scheme handle that? Hi Ashutosh, Thats a good point you mentioned. Right now we don't have a way in the templates to refer to a system attribute(we might need this as you mentioned for a temp table). We can take that up later. - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70913/#review216308 --- On July 1, 2019, 6:10 p.m., Merryle Wang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70913/ > --- > > (Updated July 1, 2019, 6:10 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Le Ma, > Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3286: Populated dynamic attribute flags for AtlasAttributes > > > Diffs > - > > addons/models/1000-Hadoop/1030-hive_model.json > 8901aa4aa86fb2802a9e9b1e65c2ff8aad8855ad > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java > d9ae9e36773146fad652a1e28fc8822bae5c8557 > intg/src/main/java/org/apache/atlas/type/AtlasStructType.java > 254eee4c9d18ab87a8862ce479fc2274227c7c29 > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > 3c53c02b48747515217c9327c98209a48ee84237 > > > Diff: https://reviews.apache.org/r/70913/diff/7/ > > > Testing > --- > > Added unit test > > > Thanks, > > Merryle Wang > >
[jira] [Created] (ATLAS-3289) Add a configuration to population dynamic attributes. If the config is disabled, read the value from the Hook
Aadarsh Jajodia created ATLAS-3289: -- Summary: Add a configuration to population dynamic attributes. If the config is disabled, read the value from the Hook Key: ATLAS-3289 URL: https://issues.apache.org/jira/browse/ATLAS-3289 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3288) Implement the DynamicAttributesEvaluator method
Aadarsh Jajodia created ATLAS-3288: -- Summary: Implement the DynamicAttributesEvaluator method Key: ATLAS-3288 URL: https://issues.apache.org/jira/browse/ATLAS-3288 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Description: We aim to implement the following methods in this Jira AtlasAttributes isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db For AtlasEntityTypes - getDynEvalAttributes() was: We aim to implement the following methods in this Jira AtlasAttributes isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db For AtlasEntityTypes - getDynEvalAttributes() dynValueEval(AtlasEntity entity) - Takes in the entity instance and evaluates the dynamic attributes for that entity > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task > Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 168h > Remaining Estimate: 168h > > We aim to implement the following methods in this Jira > AtlasAttributes > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > For AtlasEntityTypes - > getDynEvalAttributes() -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3287) Populate Dynamic Relationship Attributes
Aadarsh Jajodia created ATLAS-3287: -- Summary: Populate Dynamic Relationship Attributes Key: ATLAS-3287 URL: https://issues.apache.org/jira/browse/ATLAS-3287 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Description: We aim to implement the following methods in this Jira AtlasAttributes isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db For AtlasEntityTypes - getDynEvalAttributes() dynValueEval(AtlasEntity entity) - Takes in the entity instance and evaluates the dynamic attributes for that entity was: We aim to implement the following methods in this Jira isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task > Reporter: Aadarsh Jajodia >Assignee: Merryle Wang >Priority: Major > Original Estimate: 168h > Remaining Estimate: 168h > > We aim to implement the following methods in this Jira > AtlasAttributes > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > For AtlasEntityTypes - > getDynEvalAttributes() > dynValueEval(AtlasEntity entity) - Takes in the entity instance and evaluates > the dynamic attributes for that entity -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Description: We aim to implement the following methods in this Jira isDynAttribute() - An example of this is qualifiedName for hive_db isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task > Reporter: Aadarsh Jajodia >Priority: Major > > We aim to implement the following methods in this Jira > isDynAttribute() - An example of this is qualifiedName for hive_db > isDynAttributeEvalTrigger() - An example of this is clusterName for hive_db -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes
[ https://issues.apache.org/jira/browse/ATLAS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3286: --- Summary: Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType along with attributes which might trigger re evaluation of Dynamic Attributes (was: Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType) > Populate the dynamic attribute flag for each AtlasAttribute in each > AltasEntityType along with attributes which might trigger re evaluation of > Dynamic Attributes > - > > Key: ATLAS-3286 > URL: https://issues.apache.org/jira/browse/ATLAS-3286 > Project: Atlas > Issue Type: Sub-task > Reporter: Aadarsh Jajodia >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3286) Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType
Aadarsh Jajodia created ATLAS-3286: -- Summary: Populate the dynamic attribute flag for each AtlasAttribute in each AltasEntityType Key: ATLAS-3286 URL: https://issues.apache.org/jira/browse/ATLAS-3286 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
[
https://issues.apache.org/jira/browse/ATLAS-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aadarsh Jajodia updated ATLAS-3285:
---
Description:
An example of a Dynamic Attribute for a Hive Column looks something like this
in the hive models file
{code:java}
"options": {
"dynAttribute:qualifiedName":
"{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
}
{code}
was:
An example of a Dynamic Attribute for a Hive Column something like this
{code:java}
"options": {
"dynAttribute:qualifiedName":
"{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
}
{code}
> Defining the template version for QualifiedName in the models file for hive
> entity types
>
>
> Key: ATLAS-3285
> URL: https://issues.apache.org/jira/browse/ATLAS-3285
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Aadarsh Jajodia
>Assignee: Merryle Wang
>Priority: Major
>
> An example of a Dynamic Attribute for a Hive Column looks something like this
> in the hive models file
> {code:java}
> "options": {
> "dynAttribute:qualifiedName":
> "{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
[
https://issues.apache.org/jira/browse/ATLAS-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aadarsh Jajodia updated ATLAS-3285:
---
Description:
An example of a Dynamic Attribute for a Hive Column something like this
{code:java}
"options": {
"dynAttribute:qualifiedName":
"{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
}
{code}
> Defining the template version for QualifiedName in the models file for hive
> entity types
>
>
> Key: ATLAS-3285
> URL: https://issues.apache.org/jira/browse/ATLAS-3285
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Aadarsh Jajodia
>Assignee: Merryle Wang
>Priority: Major
>
> An example of a Dynamic Attribute for a Hive Column something like this
> {code:java}
> "options": {
> "dynAttribute:qualifiedName":
> "{table.db.name}.{table.name}.{name}@{table.db.clusterName}"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3285) Defining the template version for QualifiedName in the models file for hive entity types
Aadarsh Jajodia created ATLAS-3285: -- Summary: Defining the template version for QualifiedName in the models file for hive entity types Key: ATLAS-3285 URL: https://issues.apache.org/jira/browse/ATLAS-3285 Project: Atlas Issue Type: Sub-task Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3284) Adding functionality to compute dynamic attributes for an entity.
Aadarsh Jajodia created ATLAS-3284: -- Summary: Adding functionality to compute dynamic attributes for an entity. Key: ATLAS-3284 URL: https://issues.apache.org/jira/browse/ATLAS-3284 Project: Atlas Issue Type: New Feature Reporter: Aadarsh Jajodia This task aims at computing dynamic attributes for an entity. The first one we are going to take up in qualifiedName -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3130) Atlas stops reporting CTAS tables once an invalid relationshipDef is created
[
https://issues.apache.org/jira/browse/ATLAS-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851328#comment-16851328
]
Aadarsh Jajodia commented on ATLAS-3130:
[~umesh.padashetty] could you give us the steps to reproduce this?
[~merrylewang] tried to reproduce this but he could not.
> Atlas stops reporting CTAS tables once an invalid relationshipDef is created
> -
>
> Key: ATLAS-3130
> URL: https://issues.apache.org/jira/browse/ATLAS-3130
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Umesh Padashetty
>Priority: Minor
>
> Created an invalid relationshipDef with the following body
> {code:java}
> {
> "relationshipDefs": [{
> "propagateTags": "ONE_TO_TWO",
> "description": "default relationshipDef description with name:
> sample_dcef_KcFcR3xIN",
> "relationshipCategory": "ASSOCIATION",
> "typeVersion": "1.0",
> "attributeDefs": [],
> "endDef2": {
> "name": "inputs",
> "isLegacyAttribute": false,
> "isContainer": false,
> "cardinality": "SINGLE",
> "type": "Process",
> "description": "default relationshipEndDef description with name: inputs"
> },
> "endDef1": {
> "name": "hive_table_to_process",
> "isLegacyAttribute": false,
> "isContainer": false,
> "cardinality": "SET",
> "type": "hive_table",
> "description": "default relationshipEndDef description with name:
> hive_table_to_process"
> },
> "guid": "-302256315524233986",
> "name": "sample_dcef_KcFcR3xIN"
> }]
> }{code}
> After this, all the CTAS queries start failing with the following error
> {code:java}
> 2019-04-09 17:47:38,992 ERROR - [NotificationHookConsumer thread-0:] ~ graph
> rollback due to exception (GraphTransactionInterceptor:154)
> org.apache.atlas.exception.AtlasBaseException: invalid relationshipDef:
> sample_dcef_KcFcR3xIN: end type 1: Process, end type 2:
> hive_column_lineage{code}
> The tables are created in the hive, but they are not reported in Atlas.
> Creation of such an invalid relationshipDef should ideally be blocked.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-3235) Change the display name of hive process and hive process execution
Aadarsh Jajodia created ATLAS-3235: -- Summary: Change the display name of hive process and hive process execution Key: ATLAS-3235 URL: https://issues.apache.org/jira/browse/ATLAS-3235 Project: Atlas Issue Type: Improvement Reporter: Aadarsh Jajodia Assignee: Aadarsh Jajodia Currently the display name of a process and a process execution shows the query text. This needs to change because the query text can become too large. Either the qualifiedName or some other text needs to be shown for the display name of these -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70708: ATLAS-3185: Create process execution for Impala integration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70708/#review215511 --- Fix it, then Ship it! Overall looks good to me. Some minor comments. addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java Lines 561-562 (patched) <https://reviews.apache.org/r/70708/#comment302243> Maybe use long for both? addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/CreateImpalaProcess.java Lines 120 (patched) <https://reviews.apache.org/r/70708/#comment302242> Change to processExecution != null addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java Lines 201-205 (patched) <https://reviews.apache.org/r/70708/#comment302244> Can we also add a unit test which produces more than one execution? - Aadarsh Jajodia On May 23, 2019, 9:11 p.m., Na Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70708/ > --- > > (Updated May 23, 2019, 9:11 p.m.) > > > Review request for atlas, Aadarsh Jajodia, Madhan Neethiraj, and Sarath > Subramanian. > > > Repository: atlas > > > Description > --- > > ATLAS-3133 adds a new feature to track metadata for different executions of > the same process in Atlas. Need to add this in Impala integration > > > Diffs > - > > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/AtlasImpalaHookContext.java > 88faace > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java > 232a569 > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/BaseImpalaEvent.java > 63c5f87 > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/events/CreateImpalaProcess.java > 0dc520c > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageITBase.java > 0138d88 > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/ImpalaLineageToolIT.java > 033a518 > > addons/impala-bridge/src/test/java/org/apache/atlas/impala/hook/ImpalaLineageHookIT.java > 86801e3 > > > Diff: https://reviews.apache.org/r/70708/diff/1/ > > > Testing > --- > > Integration tests > > > Thanks, > > Na Li > >
[jira] [Commented] (ATLAS-2779) Create Hive Metastore Event listener to push table changes to Atlas.
[ https://issues.apache.org/jira/browse/ATLAS-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847785#comment-16847785 ] Aadarsh Jajodia commented on ATLAS-2779: This is duplicate of ATLAS-3148 which is resolved. Hence closing this one > Create Hive Metastore Event listener to push table changes to Atlas. > > > Key: ATLAS-2779 > URL: https://issues.apache.org/jira/browse/ATLAS-2779 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0-alpha >Reporter: Tiffany Nguyen >Priority: Major > Fix For: 1.0.0-alpha > > > Create Hive Metastore Event listener to push table changes to Atlas. This > will help push changes done via SparkSQL which does not go through hivehook > to reflect in Atlas. > Furthermore, this will eliminate the need to set hook on the client side to > capture DDL. For example, using hive CLI (by typing hive at command line) > would require changes in hive-site.xml on the client side to include hook > property (hive.exec.post.hooks). The benefit of no change needed on client > side is amplified in environment where there are hundreds of clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70600: ATLAS-3214 Create Spark Models in Atlas to Support Spark Lineage
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/#review215368 --- Hi Jungtaek, We are removing the durationMs as mentioned by you and commmiting these model changes. Thanks - Aadarsh Jajodia On May 17, 2019, 5:47 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70600/ > --- > > (Updated May 17, 2019, 5:47 p.m.) > > > Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, > Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > This is an initial mode file for the SPARK models. Please refer to the file > added here. This file is a GET call on typeDefs on a cluster which had SAC > enabled and SAC has created the models. The SAC version used was > https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev > > > Diffs > - > > addons/models/1000-Hadoop/1100-spark_model.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/70600/diff/4/ > > > Testing > --- > > > File Attachments > > > SAC Models > > https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt > > > Thanks, > > Aadarsh Jajodia > >
Re: Review Request 70600: ATLAS-3214 Create Spark Models in Atlas to Support Spark Lineage
> On May 17, 2019, 4:28 a.m., Jungtaek Lim wrote: > > > > Jungtaek Lim wrote: > > Removing isLegacy Attributes since spark is not released yet. All > fields that are referenced will show up in the relations tab. > > I think we've talked yesterday and decided to keep the flag on since we > want these attributes to also show up in properties tab. For example, see > `spark_ml_pipeline`. If we can't show up these attributes in properties tab, > end users have to do couple of clicks to see these value in relationship tab, > and properties tab is no longer working as `overview`. Hi Jungtaek, Traditionally Atlas used to follow that. But with recent changes to Atlas, all relationship attributes will show up on the relationship tab only. For any new models that is the convention. Thank you - Aadarsh --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/#review215314 ------- On May 16, 2019, 11:54 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70600/ > --- > > (Updated May 16, 2019, 11:54 p.m.) > > > Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, > Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > This is an initial mode file for the SPARK models. Please refer to the file > added here. This file is a GET call on typeDefs on a cluster which had SAC > enabled and SAC has created the models. The SAC version used was > https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev > > > Diffs > - > > addons/models/1000-Hadoop/1100-spark_model.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/70600/diff/3/ > > > Testing > --- > > > File Attachments > > > SAC Models > > https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt > > > Thanks, > > Aadarsh Jajodia > >
[jira] [Comment Edited] (ATLAS-3100) Enhance Full Text Search backend implementation to use SOLR request handler.
[ https://issues.apache.org/jira/browse/ATLAS-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841746#comment-16841746 ] Aadarsh Jajodia edited comment on ATLAS-3100 at 5/16/19 10:02 PM: -- Guidelines followed for adding Search Weight * 10: name, displayName, fullName, topic * 9: description, comment, userName, owner, queryText, owners, doc * 8: clusterName, namespace, hostName, uri, contactInfo, pipelines, avroclass, namespace, domain, region subdomain * 5: for all other searchable attributes was (Author: django): Guidelines followed for adding Search Weight * 10: name, displayName, fullName, topic * 9: description, comment, userName, owner, queryText * 8: clusterName, namespace, hostName, uri, contactInfo * 5: for all other searchable attributes > Enhance Full Text Search backend implementation to use SOLR request handler. > > > Key: ATLAS-3100 > URL: https://issues.apache.org/jira/browse/ATLAS-3100 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.1.0 >Reporter: Sridhar >Assignee: Sridhar >Priority: Major > Attachments: Atlas Improvements--Support for Free text search, > Auto-Suggestion and Faceting..pdf > > > For full text search, Atlas is using separate collection with name > "Full-Text-Search" for text searches. This collection maintains index on a > Full Text string that is created using all the attributes of the Entity. This > improvement will try to eliminate the need to maintain this collection and > use SOLR search more efficiently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3100) Enhance Full Text Search backend implementation to use SOLR request handler.
[ https://issues.apache.org/jira/browse/ATLAS-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841746#comment-16841746 ] Aadarsh Jajodia commented on ATLAS-3100: Guidelines followed for adding Search Weight * 10: name, displayName, fullName, topic * 9: description, comment, userName, owner, queryText * 8: clusterName, namespace, hostName, uri, contactInfo * 5: for all other searchable attributes > Enhance Full Text Search backend implementation to use SOLR request handler. > > > Key: ATLAS-3100 > URL: https://issues.apache.org/jira/browse/ATLAS-3100 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.1.0 >Reporter: Sridhar >Assignee: Sridhar >Priority: Major > Attachments: Atlas Improvements--Support for Free text search, > Auto-Suggestion and Faceting..pdf > > > For full text search, Atlas is using separate collection with name > "Full-Text-Search" for text searches. This collection maintains index on a > Full Text string that is created using all the attributes of the Entity. This > improvement will try to eliminate the need to maintain this collection and > use SOLR search more efficiently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3214) Create Spark Models in Atlas to support Spark Lineage
Aadarsh Jajodia created ATLAS-3214: -- Summary: Create Spark Models in Atlas to support Spark Lineage Key: ATLAS-3214 URL: https://issues.apache.org/jira/browse/ATLAS-3214 Project: Atlas Issue Type: New Feature Reporter: Aadarsh Jajodia Assignee: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70600: Adding the model files for Spark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/ --- (Updated May 16, 2019, 6:24 p.m.) Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Renaming the file to 1100-spark model since we already have a 1090 version for Imapala. Also added the attribtues suggested by Jungtaek. Have added relationship label as suggested by Madhan, when both ends of a relationship have isLegacyAttribtue set to true. Have also changed the type of executionTime field to date Repository: atlas Description --- This is an initial mode file for the SPARK models. Please refer to the file added here. This file is a GET call on typeDefs on a cluster which had SAC enabled and SAC has created the models. The SAC version used was https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev Diffs (updated) - addons/models/1000-Hadoop/1100-spark_model.json PRE-CREATION Diff: https://reviews.apache.org/r/70600/diff/2/ Changes: https://reviews.apache.org/r/70600/diff/1-2/ Testing --- File Attachments SAC Models https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt Thanks, Aadarsh Jajodia
Re: Review Request 70600: Adding the model files for Spark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/#review215266 --- addons/models/1000-Hadoop/1090-spark_model.json Lines 22 (patched) <https://reviews.apache.org/r/70600/#comment301908> This should not be optional. If you refer to the corresponding hive model, it is not optional addons/models/1000-Hadoop/1090-spark_model.json Lines 49 (patched) <https://reviews.apache.org/r/70600/#comment301913> Jungtaek, if you look at the spark table and see the differences between spark and hive table and let us know if these differences are expected and fine. addons/models/1000-Hadoop/1090-spark_model.json Lines 139 (patched) <https://reviews.apache.org/r/70600/#comment301909> In the corresponding hive_column models i see there is something called a position attribute. Are we expecting to not have that attribute for Spark Column addons/models/1000-Hadoop/1090-spark_model.json Lines 207 (patched) <https://reviews.apache.org/r/70600/#comment301911> I just wanted to highlight that i see some differences between spark storage desc and hive storage desc in the attributes addons/models/1000-Hadoop/1090-spark_model.json Lines 249 (patched) <https://reviews.apache.org/r/70600/#comment301910> Why is this set to true here? Any particular reason? addons/models/1000-Hadoop/1090-spark_model.json Lines 313 (patched) <https://reviews.apache.org/r/70600/#comment301915> This should be changed to date type. I will update in the next version - Aadarsh Jajodia On May 10, 2019, 9:18 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70600/ > --- > > (Updated May 10, 2019, 9:18 p.m.) > > > Review request for atlas, Ankur Gupta, Ashutosh Mestry, Jungtaek Lim, Sridhar > K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > This is an initial mode file for the SPARK models. Please refer to the file > added here. This file is a GET call on typeDefs on a cluster which had SAC > enabled and SAC has created the models. The SAC version used was > https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev > > > Diffs > - > > addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/70600/diff/1/ > > > Testing > --- > > > File Attachments > > > SAC Models > > https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt > > > Thanks, > > Aadarsh Jajodia > >
[jira] [Assigned] (ATLAS-3209) Add unit tests to verify no process and process executions are created by HMS Hook
[ https://issues.apache.org/jira/browse/ATLAS-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia reassigned ATLAS-3209: -- Assignee: Aadarsh Jajodia > Add unit tests to verify no process and process executions are created by HMS > Hook > -- > > Key: ATLAS-3209 > URL: https://issues.apache.org/jira/browse/ATLAS-3209 > Project: Atlas > Issue Type: Test > Reporter: Aadarsh Jajodia > Assignee: Aadarsh Jajodia >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3209) Add unit tests to verify no process and process executions are created by HMS Hook
Aadarsh Jajodia created ATLAS-3209: -- Summary: Add unit tests to verify no process and process executions are created by HMS Hook Key: ATLAS-3209 URL: https://issues.apache.org/jira/browse/ATLAS-3209 Project: Atlas Issue Type: Test Reporter: Aadarsh Jajodia -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70600: Adding the model files for Spark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/#review215207 --- Hi Jungtaek, I spoke to Ankur on Friday and he mentions all spark tables/db/cols in CDP will always be created in HMS. If that is the case we should remove spark tables/db/cols models. In case spark will use Hbase to create thsee then we should map them to hbase models that we have in Atlas. Ankur could you please comment on the same. Thanks - Aadarsh Jajodia On May 10, 2019, 9:18 p.m., Aadarsh Jajodia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70600/ > --- > > (Updated May 10, 2019, 9:18 p.m.) > > > Review request for atlas, Ankur Gupta, Ashutosh Mestry, Jungtaek Lim, Sridhar > K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > This is an initial mode file for the SPARK models. Please refer to the file > added here. This file is a GET call on typeDefs on a cluster which had SAC > enabled and SAC has created the models. The SAC version used was > https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev > > > Diffs > - > > addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/70600/diff/1/ > > > Testing > --- > > > File Attachments > > > SAC Models > > https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt > > > Thanks, > > Aadarsh Jajodia > >
Re: Review Request 70605: ATLAS-3197 Add QueryText for hive_table and hive_db
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70605/#review215147 --- If i run a CTAS query, does the newly created table get any hive_ddl attritbutes? - Aadarsh Jajodia On May 8, 2019, 11:22 p.m., Le Ma wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70605/ > --- > > (Updated May 8, 2019, 11:22 p.m.) > > > Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, Sridhar K, Madhan > Neethiraj, and Sarath Subramanian. > > > Bugs: ATLAS-3197 > https://issues.apache.org/jira/browse/ATLAS-3197 > > > Repository: atlas > > > Description > --- > > This patch introduced a model called Hive_table_ddl and hive_db_ddl which are > used to track commands that are executed either on a table entity or bd > entity. DDL entities will be added to a table/db entity through > relationshipAttribute call HIVE_DB_TO_DDL and hive_tb_to_ddl. Every time when > a new command is executed, a new ddl entity will be created, it will append > to the existing DDLQueries field if a table/db has already been created. Once > a table/db is deleted, all these relationship attribute will be deleted as > well. > > > Diffs > - > > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java > d3d83490f > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java > de8de63b6 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateDatabase.java > b01f61f76 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java > b5ce09332 > addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java > 24de30f5a > addons/models/1000-Hadoop/1030-hive_model.json ee6945a77 > > > Diff: https://reviews.apache.org/r/70605/diff/4/ > > > Testing > --- > > 1. table : > > CREATE TABLE IF NOT EXISTS testDDL (d int, n string); > > > ALTER TABLE testDDL RENAME TO renameTestDDL; > > ALTER TABLE renameTestDDL CHANGE n changedCol string; > > ALTER TABLE renameTestDDL ADD COLUMNS(addedCol int); > > DROP TABLE renameTestDDL; > > > 2. db > > create database testdb; > > ALTER DATABASE database SET OWNER USER changedOwner; > > DROP DATABSE testdb; > > > File Attachments > > > CREATE TABLE IF NOT EXISTS testDDL.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/5047d461-f288-4776-a843-556e45217fef__CREATE_TABLE_IF_NOT_EXISTS_testDDL.png > ALTER TABLE testDDL RENAME TO renameTestDDL; .png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/373a656d-8ab3-4710-b9a1-3eac20adc947__ALTER_TABLE_testDDL_RENAME_TO_renameTestDDL_.png > ALTER TABLE renameTestDDL CHANGE n changedCol string; - ddl.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/c9f57d67-c16b-4142-a6b4-9d372e7fd765__ALTER_TABLE_renameTestDDL_CHANGE_n_changedCol_string_-_ddl.png > ALTER TABLE renameTestDDL CHANGE n changedCol string;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/42612eb6-83c2-4825-8601-5a1629c9de6f__ALTER_TABLE_renameTestDDL_CHANGE_n_changedCol_string.png > ALTER TABLE renameTestDDL ADD COLUMNS(addedCol int);.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/f5d3522d-17b6-4227-8072-6fc07d88d515__ALTER_TABLE_renameTestDDL_ADD_COLUMNSaddedCol_int.png > ALTER TABLE renameTestDDL ADD COLUMNS.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/73b9046a-2667-4049-9e11-c5c2b375e9ca__ALTER_TABLE_renameTestDDL_ADD_COLUMNS.png > DROP TABLE renameTestDDL;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/ebfa01f2-e886-4f94-aa75-ff45bb519ca0__DROP_TABLE_renameTestDDL.png > ddl - relationship.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/7b30282b-6419-4791-b107-f938918e1cfd__ddl_-_relationship.png > ALTER DATABASE testdb SET OWNER USER changedOwner;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/388e7ffa-6691-4c2b-bcd9-d8755dc13e3e__ALTER_DATABASE_testdb_SET_OWNER_USER_changedOwner.png > ALTER DATABASE testdb SET OWNER USER changedOwner; - db entity.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/fd22b6a5-6cb9-40eb-aa10-5389f6f74792__ALTER_DATABASE_testdb_SET_OWNER_USER_changedOwner_-_db_entity.png > drop database testdb;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/7c889d62-66c8-4c26-bf94-257bd01cdb70__drop_database_testdb.png > drop database testdb;.png > > https://reviews.apache.org/media/uploaded/files/2019/05/07/4163043b-c2f1-42ee-9e5c-cfffbc589fb3__drop_database_testdb.png > > > Thanks, > > Le Ma > >
Re: Review Request 70512: ATLAS-3183: Read Impala lineage record for creating view and send to Atlas
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70512/#review215078
---
addons/impala-bridge/src/main/java/org.apache.atlas.impala/ImpalaLineageTool.java
Lines 64 (patched)
<https://reviews.apache.org/r/70512/#comment301491>
Replace with
LOG.info("Importing: {}", filename);
addons/impala-bridge/src/main/java/org.apache.atlas.impala/ImpalaLineageTool.java
Lines 65 (patched)
<https://reviews.apache.org/r/70512/#comment301492>
What are we doing with the return value of this method?
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/BaseImpalaEvent.java
Lines 118 (patched)
<https://reviews.apache.org/r/70512/#comment301480>
What if node itself is null?
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 158 (patched)
<https://reviews.apache.org/r/70512/#comment301482>
Do we need this check? We can just put LOG.debug. If log level is set to
debug it will log it else it will not isnt it?
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 165 (patched)
<https://reviews.apache.org/r/70512/#comment301490>
Remove extra line
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 171 (patched)
<https://reviews.apache.org/r/70512/#comment301489>
Remove extra line
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 189 (patched)
<https://reviews.apache.org/r/70512/#comment301488>
Remove extra line
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 203 (patched)
<https://reviews.apache.org/r/70512/#comment301481>
Change ":" to constant
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 227 (patched)
<https://reviews.apache.org/r/70512/#comment301483>
Do we need the isDebugEnabled check? We can just put LOG.debug. If log
level is set to debug it will log it else it will not isnt it?
Do we also need the isColumnLineage check? I see that above we are creating
objects of columnlineage and hence they will never be NULL. I feel this check
can be avoided here
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 236 (patched)
<https://reviews.apache.org/r/70512/#comment301493>
Add comment for method
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 262 (patched)
<https://reviews.apache.org/r/70512/#comment301486>
Add comment for method
addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/events/CreateImpalaProcess.java
Lines 304-310 (patched)
<https://reviews.apache.org/r/70512/#comment301484>
Can be simplified
if (inputNodes.isEmpty() || ouputNodes.isEmpty()) {
return true;
}
addons/impala-bridge/src/main/java/org.apache.atlas.impala/model/ImpalaDependencyType.java
Lines 21 (patched)
<https://reviews.apache.org/r/70512/#comment301485>
extra space after enum
- Aadarsh Jajodia
On May 4, 2019, 9:23 p.m., Na Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70512/
> -------
>
> (Updated May 4, 2019, 9:23 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, madhan, Sarath
> Subramanian, and Xinran Tinney.
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Impala generates lineage records for its commands. This new feature will read
> Impala lineage file, convert the lineage record to Atlas entities and send
> them to Atlas. In this way, Atlas can get lineage of Impala operation.
>
> The metadata referred in the lineage are captured in Hive Metastore hook and
> sent to Atlas. This work is done in ATLAS-3148
>
> This jira only supports the Impala command "create view". Following jira will
> add support for more Impala commands.
>
>
> Diffs
> -
>
> addons/impala-bridge/pom.xml PRE-CREATION
>
> addons/impala-bridge/src/main/java/org.apache.atlas.impala/ImpalaLineageTool.java
> PRE-CREATION
>
> addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/AtlasImpalaHookContext.java
> PRE-CREATION
>
> addons/impala-bridge/src/main/java/org.apache.atlas.impala/hook/ImpalaIdentifierParser.java
> PRE-CREATION
>
> addons/impala-bri
Re: Review Request 70600: Adding the model files for Spark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/ --- (Updated May 7, 2019, 1:32 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description (updated) --- This is an initial mode file for the SPARK models. Please refer to the file added here. This file is a GET call on typeDefs on a cluster which had SAC enabled and SAC has created the models. The SAC version used was https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev Diffs - addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION Diff: https://reviews.apache.org/r/70600/diff/1/ Testing --- File Attachments SAC Models https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt Thanks, Aadarsh Jajodia
Review Request 70600: Adding the model files for Spark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70600/ --- Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Repository: atlas Description --- Adding the model files for Spark Diffs - addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION Diff: https://reviews.apache.org/r/70600/diff/1/ Testing --- File Attachments SAC Models https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt Thanks, Aadarsh Jajodia
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: Adding Process Executions In Atlas(Community).pdf > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch, Adding > Process Executions In Atlas(Community).pdf > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16825587#comment-16825587 ] Aadarsh Jajodia commented on ATLAS-3133: Here is the review request for the same [https://reviews.apache.org/r/70462/] > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: (was: 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch) > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: (was: 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch) > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: (was: 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch) > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70462: ATLAS-3133 : Adding support for Process Executions in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70462/ --- (Updated April 23, 2019, 9:37 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Tested these changes on a VM and it works now. Bugs: ATLAS-3133 https://issues.apache.org/jira/browse/ATLAS-3133 Repository: atlas Description --- Adding support for Process Executions in Atlas Diffs (updated) - addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java 52da7106282bc9ab0c8a99bccd47d7227867ec70 addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 6109297e30895f9329b82295e520f577aaa6954b addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java 4127c611a271bf3cbfc747cf1ac23ecd120d8a50 addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java 019deb73c9c875033c1bc1117fb3188fd9749af2 addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java b6ec5c330bed988e9f66298a68881f5fadb079f8 addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataTypes.java 94010d0cb1a7a5c48b71f6d77c5e1a8f5cfcf013 addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java c8c53c473db02bdb523b91f71d3c3d10ceb57c5a addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 677043a01bbebf0fe998feae49e6af6915685fad addons/models/-Area0/0010-base_model.json 8b41ee89f3a7288bb4cdad3df6887242b40b68d2 addons/models/1000-Hadoop/1030-hive_model.json e96443382a587411d1207d6e1157ea65350cbdba webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6fcea9ed43f7e3d93742c7de4bc4db4386ecdb2d Diff: https://reviews.apache.org/r/70462/diff/8/ Changes: https://reviews.apache.org/r/70462/diff/7-8/ Testing --- We want to add support for Process Executions in Atlas. With the help of process executions attributes specific to each execution(like startTime, endTime, queryText) will be captured in the execution object. With the current model of Atlas this is lost since each execution overrides the previous one. To solve this problem. We are creating 2 new entity definitions and one relations definition. We create an entity definition called a process execution and another one called hive process execution. The hive process execution has all the attributes which are specific to each execution. We also create a relationship defintion between a hive process and a hive process execution as a many to one relation. The lineage does not get affected in this since the inputs and outputs are still attached to the hive process. Hive Process can now be thought of as a grouping of multiple executions. There is no lineage shown for a hive process execution. The criteria for grouping is based on the qualifie dName of a Hive Process. As long as the qualified Name of a hive process remains the same, each execution gets mapped to the same hive process. Thanks, Aadarsh Jajodia
[jira] [Commented] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16823535#comment-16823535 ] Aadarsh Jajodia commented on ATLAS-3133: Updated Latest Patch > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch, > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch, > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3133: --- Attachment: (was: 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch) > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > Attachments: > 0001-ATLAS-3133-Adding-support-for-Process-Executions-in-.patch > > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run same DDL (e.g. insert into table A select * from table B, C) multiple > times Atlas does not capture the multiple instances of execution of the same > logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70462: ATLAS-3133 : Adding support for Process Executions in Atlas
> On April 17, 2019, 9:08 p.m., Sarath Subramanian wrote:
> > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
> > Lines 144 (patched)
> > <https://reviews.apache.org/r/70462/diff/5/?file=2139793#file2139793line145>
> >
> > change DEBUG to WARN, also you are missing a {} in the debug message.
> > consider moving the logic to get hostname in getHostName() method
Changing message level to warning.
- Aadarsh
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70462/#review214732
-------
On April 16, 2019, 6:06 p.m., Aadarsh Jajodia wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70462/
> ---
>
> (Updated April 16, 2019, 6:06 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan
> Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3133
> https://issues.apache.org/jira/browse/ATLAS-3133
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Adding support for Process Executions in Atlas
>
>
> Diffs
> -
>
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
> 44c6437fe8e9bf93d823157ca0ff7b36cec9bb87
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
> ee02285df67aa27f45f1116fd0c2015b779f9311
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
> 31346d0954140cd8bda690dc9079e0913f7b9d7d
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java
> d61f1d74e3238e0a7474de67c0400c108d8919ea
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java
> 674a89f6e4852dc30c29c5681854ec3ba8611f35
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataTypes.java
> 94010d0cb1a7a5c48b71f6d77c5e1a8f5cfcf013
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java
> 002b90839f78dc843b5aca56042c3decd299bed8
> addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java
> 142e35561fff42f7672c5b5136af1173049580b3
> addons/models/-Area0/0010-base_model.json
> 8b41ee89f3a7288bb4cdad3df6887242b40b68d2
> addons/models/1000-Hadoop/1030-hive_model.json
> e96443382a587411d1207d6e1157ea65350cbdba
> webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java
> 6fcea9ed43f7e3d93742c7de4bc4db4386ecdb2d
>
>
> Diff: https://reviews.apache.org/r/70462/diff/5/
>
>
> Testing
> ---
>
> We want to add support for Process Executions in Atlas. With the help of
> process executions attributes specific to each execution(like startTime,
> endTime, queryText) will be captured in the execution object. With the
> current model of Atlas this is lost since each execution overrides the
> previous one. To solve this problem. We are creating 2 new entity definitions
> and one relations definition. We create an entity definition called a process
> execution and another one called hive process execution. The hive process
> execution has all the attributes which are specific to each execution. We
> also create a relationship defintion between a hive process and a hive
> process execution as a many to one relation. The lineage does not get
> affected in this since the inputs and outputs are still attached to the hive
> process. Hive Process can now be thought of as a grouping of multiple
> executions. There is no lineage shown for a hive process execution. The
> criteria for grouping is based on the qualif
iedName of a Hive Process. As long as the qualified Name of a hive process
remains the same, each execution gets mapped to the same hive process.
>
>
> Thanks,
>
> Aadarsh Jajodia
>
>
Re: Review Request 70462: ATLAS-3133 : Adding support for Process Executions in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70462/ --- (Updated April 15, 2019, 10:31 p.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Adding changes for the quick start. Also added changes for the missing unit test Bugs: ATLAS-3133 https://issues.apache.org/jira/browse/ATLAS-3133 Repository: atlas Description --- Adding support for Process Executions in Atlas Diffs (updated) - addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java 44c6437fe8e9bf93d823157ca0ff7b36cec9bb87 addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java ee02285df67aa27f45f1116fd0c2015b779f9311 addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java 31346d0954140cd8bda690dc9079e0913f7b9d7d addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java d61f1d74e3238e0a7474de67c0400c108d8919ea addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java 674a89f6e4852dc30c29c5681854ec3ba8611f35 addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataTypes.java 94010d0cb1a7a5c48b71f6d77c5e1a8f5cfcf013 addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 002b90839f78dc843b5aca56042c3decd299bed8 addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 142e35561fff42f7672c5b5136af1173049580b3 addons/models/-Area0/0010-base_model.json 8b41ee89f3a7288bb4cdad3df6887242b40b68d2 addons/models/1000-Hadoop/1030-hive_model.json e96443382a587411d1207d6e1157ea65350cbdba webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 6fcea9ed43f7e3d93742c7de4bc4db4386ecdb2d Diff: https://reviews.apache.org/r/70462/diff/4/ Changes: https://reviews.apache.org/r/70462/diff/3-4/ Testing --- We want to add support for Process Executions in Atlas. With the help of process executions attributes specific to each execution(like startTime, endTime, queryText) will be captured in the execution object. With the current model of Atlas this is lost since each execution overrides the previous one. To solve this problem. We are creating 2 new entity definitions and one relations definition. We create an entity definition called a process execution and another one called hive process execution. The hive process execution has all the attributes which are specific to each execution. We also create a relationship defintion between a hive process and a hive process execution as a many to one relation. The lineage does not get affected in this since the inputs and outputs are still attached to the hive process. Hive Process can now be thought of as a grouping of multiple executions. There is no lineage shown for a hive process execution. The criteria for grouping is based on the qualifie dName of a Hive Process. As long as the qualified Name of a hive process remains the same, each execution gets mapped to the same hive process. Thanks, Aadarsh Jajodia
Re: Review Request 70462: ATLAS-3133 : Adding support for Process Executions in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70462/ --- (Updated April 12, 2019, 12:39 a.m.) Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian. Changes --- Adding the Atlas JIRA to the review Summary (updated) - ATLAS-3133 : Adding support for Process Executions in Atlas Bugs: ATLAS-3133 https://issues.apache.org/jira/browse/ATLAS-3133 Repository: atlas Description --- Adding support for Process Executions in Atlas Diffs (updated) - addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java 31346d0954140cd8bda690dc9079e0913f7b9d7d addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateHiveProcess.java d61f1d74e3238e0a7474de67c0400c108d8919ea addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java 674a89f6e4852dc30c29c5681854ec3ba8611f35 addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataTypes.java 94010d0cb1a7a5c48b71f6d77c5e1a8f5cfcf013 addons/hive-bridge/src/test/java/org/apache/atlas/hive/HiveITBase.java 002b90839f78dc843b5aca56042c3decd299bed8 addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java 142e35561fff42f7672c5b5136af1173049580b3 addons/models/-Area0/0010-base_model.json 8b41ee89f3a7288bb4cdad3df6887242b40b68d2 addons/models/1000-Hadoop/1030-hive_model.json e96443382a587411d1207d6e1157ea65350cbdba Diff: https://reviews.apache.org/r/70462/diff/2/ Changes: https://reviews.apache.org/r/70462/diff/1-2/ Testing --- We want to add support for Process Executions in Atlas. With the help of process executions attributes specific to each execution(like startTime, endTime, queryText) will be captured in the execution object. With the current model of Atlas this is lost since each execution overrides the previous one. To solve this problem. We are creating 2 new entity definitions and one relations definition. We create an entity definition called a process execution and another one called hive process execution. The hive process execution has all the attributes which are specific to each execution. We also create a relationship defintion between a hive process and a hive process execution as a many to one relation. The lineage does not get affected in this since the inputs and outputs are still attached to the hive process. Hive Process can now be thought of as a grouping of multiple executions. There is no lineage shown for a hive process execution. The criteria for grouping is based on the qualifie dName of a Hive Process. As long as the qualified Name of a hive process remains the same, each execution gets mapped to the same hive process. Thanks, Aadarsh Jajodia
[jira] [Assigned] (ATLAS-3133) Capture metadata for different executions of the same process in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia reassigned ATLAS-3133: -- Assignee: Aadarsh Jajodia > Capture metadata for different executions of the same process in Atlas > -- > > Key: ATLAS-3133 > URL: https://issues.apache.org/jira/browse/ATLAS-3133 > Project: Atlas > Issue Type: New Feature > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Assignee: Aadarsh Jajodia >Priority: Critical > > Background: The current Atlas process metadata model within Atlas does not > track multiple instances of execution of the same process. For example if we > run a CTAS command (e.g. insert into table A select * from table B, C) > multiple times Atlas does not capture the multiple instances of execution of > the same logical process. > User story: As a compliance admin or data steward, I need to be able to track > multiple executions of the same process or pipeline that were done at > different times and relate them to the logical process so that I can provide > traceability and understand how different instances of my data pipelining > operations performed over time. > Acceptance Criteria: > 1) Every new instance of process execution is captured with the appropriate > metadata for the process along with context (who, what, when) > 2) One can navigate between process and process execution in Atlas UI and > explore relevant metadata > 3) Process nodes in lineage display high level information about number of > process executions associated with a particular process node. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3041) Cannot delete relationship types using the types/typedef/name REST API
[ https://issues.apache.org/jira/browse/ATLAS-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3041: --- Attachment: (was: ATLAS-3041.patch) > Cannot delete relationship types using the types/typedef/name REST API > -- > > Key: ATLAS-3041 > URL: https://issues.apache.org/jira/browse/ATLAS-3041 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Chris Clohosy >Assignee: Aadarsh Jajodia >Priority: Major > Attachments: ATLAS-3041.patch > > Time Spent: 40m > Remaining Estimate: 0h > > It appears that relationship types cannot be deleted by doing a HTTP DELETE > to the types/typedef/name REST API endpoint. Upon further exploration of the > source code (org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore) > the problem seems to lie within the deleteTypeByName method - this calls the > getByName method and from logging we can observe it correctly gets back the > AtlasRelationshipType. However there is no subsequent check for this type > within deleteTypeByName, and as AtlasRelationshipTime extends AtlasStructType > the code then thinks it needs to delete a struct instead of a relationship > and therefore fails. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3041) Cannot delete relationship types using the types/typedef/name REST API
[ https://issues.apache.org/jira/browse/ATLAS-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3041: --- Attachment: ATLAS-3041.patch > Cannot delete relationship types using the types/typedef/name REST API > -- > > Key: ATLAS-3041 > URL: https://issues.apache.org/jira/browse/ATLAS-3041 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Chris Clohosy >Assignee: Aadarsh Jajodia >Priority: Major > Attachments: ATLAS-3041.patch > > Time Spent: 40m > Remaining Estimate: 0h > > It appears that relationship types cannot be deleted by doing a HTTP DELETE > to the types/typedef/name REST API endpoint. Upon further exploration of the > source code (org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore) > the problem seems to lie within the deleteTypeByName method - this calls the > getByName method and from logging we can observe it correctly gets back the > AtlasRelationshipType. However there is no subsequent check for this type > within deleteTypeByName, and as AtlasRelationshipTime extends AtlasStructType > the code then thinks it needs to delete a struct instead of a relationship > and therefore fails. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3041) Cannot delete relationship types using the types/typedef/name REST API
[ https://issues.apache.org/jira/browse/ATLAS-3041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aadarsh Jajodia updated ATLAS-3041: --- Attachment: ATLAS-3041.patch > Cannot delete relationship types using the types/typedef/name REST API > -- > > Key: ATLAS-3041 > URL: https://issues.apache.org/jira/browse/ATLAS-3041 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Chris Clohosy >Assignee: Aadarsh Jajodia >Priority: Major > Attachments: ATLAS-3041.patch > > Time Spent: 10m > Remaining Estimate: 0h > > It appears that relationship types cannot be deleted by doing a HTTP DELETE > to the types/typedef/name REST API endpoint. Upon further exploration of the > source code (org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore) > the problem seems to lie within the deleteTypeByName method - this calls the > getByName method and from logging we can observe it correctly gets back the > AtlasRelationshipType. However there is no subsequent check for this type > within deleteTypeByName, and as AtlasRelationshipTime extends AtlasStructType > the code then thinks it needs to delete a struct instead of a relationship > and therefore fails. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
