----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/38393/#review99486 -----------------------------------------------------------
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java (line 277) <https://reviews.apache.org/r/38393/#comment156363> is sdQualifiedName supposed to be same as tableQualifiedName ? - Suma Shivaprasad On Sept. 15, 2015, 6:56 a.m., Shwetha GS wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/38393/ > ----------------------------------------------------------- > > (Updated Sept. 15, 2015, 6:56 a.m.) > > > Review request for atlas. > > > Bugs: ATLAS-58 > https://issues.apache.org/jira/browse/ATLAS-58 > > > Repository: atlas > > > Description > ------- > > Hive hook sends notification messages (list of entities). The Notification > consumer on server side consumes these messages and registers the entities. > The server handles de-duping of entities based on the unique attribute of the > entity > Big changes: > 1. Concept of service that are started and stopped at atlas start and stop > 2. De-duping of entities on server based on any unique attribute for the > entity. If entity doesn't have any unique attribute, de-duping is not done > and new entity is created > 3. Changed entity submit API to take list of entities instead of just 1 > entity (required for hive hook) - backward incompatible > 4. Moved submit and list from EntityResource to EntitiesResource - backward > incompatible > 5. Moved security tests from integration tests to unit tests - as they were > creating issues with server start as jetty already starts another server for > integration tests > 6. Removed some duplicate tests from repository module (the same tests exist > in typesystem module as well) > 7. In webapp ITs, re-used the types defined > 8. Hive hook now sends notifications instead of registering entities. Sending > notification is done synchronously. So, this adds to hive command execution > delay. But this also makes it reliable > > Pending: > 1. Entity updates like alter table commands are not handlded. Will create > another jira for this > 2. Webapp jetty plugin doesn't shutdown embedded kafka at the end of > integration tests. So, hive bridge ITs fail. Hive bridge ITs pass if run on > their own. Still checking on this > > > Diffs > ----- > > addons/hive-bridge/pom.xml 914d8c6 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java > 4d1af4e > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java > fe26446 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataModelGenerator.java > cb8359e > addons/hive-bridge/src/site/twiki/Bridge-Hive.twiki 647af27 > addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java > 2bf3aba > client/pom.xml 2e27930 > client/src/main/java/org/apache/atlas/ApplicationProperties.java 738ec53 > client/src/main/java/org/apache/atlas/AtlasClient.java 1c7d62a > client/src/test/java/org/apache/atlas/ApplicationPropertiesTest.java > 4acb3e1 > client/src/test/resources/application.properties dbd6002 > common/pom.xml PRE-CREATION > common/src/main/java/org/apache/atlas/service/Service.java PRE-CREATION > common/src/main/java/org/apache/atlas/service/Services.java PRE-CREATION > dashboard/public/modules/details/detailsResource.js af2f249 > distro/pom.xml d42d78f > distro/src/conf/application.properties bf323a7 > docs/src/site/twiki/InstallationSteps.twiki 3d0351c > notification/pom.xml b036855 > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java > 9978275 > > notification/src/main/java/org/apache/atlas/kafka/KafkaNotificationProvider.java > PRE-CREATION > > notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java > 36a62f0 > > notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java > 0951124 > > notification/src/main/java/org/apache/atlas/notification/NotificationModule.java > db17e35 > > notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java > 02752dc > pom.xml e8d18a2 > repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java > 3d47293 > repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java > fbd01de > > repository/src/main/java/org/apache/atlas/listener/EntityChangeListener.java > f58d6de > > repository/src/main/java/org/apache/atlas/repository/EntityExistsException.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java > 1b0faa5 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > cc10ea3 > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > ceb8a65 > > repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java > 56168db > repository/src/main/java/org/apache/atlas/services/MetadataService.java > 8f82f88 > repository/src/test/java/org/apache/atlas/RepositoryServiceLoadingTest.java > 18b1d1b > repository/src/test/java/org/apache/atlas/TestUtils.java de41793 > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > 127e6d7 > > repository/src/test/java/org/apache/atlas/discovery/HiveLineageServiceTest.java > 2c7d61a > repository/src/test/java/org/apache/atlas/repository/BaseTest.java 8924c37 > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java > 487ac84 > > repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java > be99efe > repository/src/test/java/org/apache/atlas/repository/memory/ClassTest.java > ed747b2 > repository/src/test/java/org/apache/atlas/repository/memory/EnumTest.java > 4ce1085 > > repository/src/test/java/org/apache/atlas/repository/memory/InstanceE2ETest.java > 3c29bf8 > > repository/src/test/java/org/apache/atlas/repository/memory/StorageTest.java > c329caf > repository/src/test/java/org/apache/atlas/repository/memory/StructTest.java > 3ab7020 > repository/src/test/java/org/apache/atlas/repository/memory/TraitTest.java > bfdc8b1 > > repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java > PRE-CREATION > repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala > eef13df > repository/src/test/scala/org/apache/atlas/query/QueryTestsUtils.scala > ec21886 > typesystem/pom.xml e4a1383 > typesystem/src/main/java/org/apache/atlas/ApplicationProperties.java > PRE-CREATION > typesystem/src/main/java/org/apache/atlas/TypeExistsException.java > PRE-CREATION > > typesystem/src/main/java/org/apache/atlas/typesystem/persistence/StructInstance.java > 309ab11 > > typesystem/src/main/java/org/apache/atlas/typesystem/types/AttributeDefinition.java > 31b5f47 > > typesystem/src/main/java/org/apache/atlas/typesystem/types/AttributeInfo.java > e74f8d5 > > typesystem/src/main/java/org/apache/atlas/typesystem/types/ObjectGraphWalker.java > 399902c > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java > d1e8dbd > > typesystem/src/main/java/org/apache/atlas/typesystem/types/ValueConversionException.java > 5cdee8d > typesystem/src/main/resources/application.properties f7e2774 > typesystem/src/main/resources/atlas-log4j.xml PRE-CREATION > > typesystem/src/main/scala/org/apache/atlas/typesystem/json/TypesSerialization.scala > 7a681bf > typesystem/src/test/java/org/apache/atlas/ApplicationPropertiesTest.java > PRE-CREATION > > typesystem/src/test/java/org/apache/atlas/typesystem/json/SerializationJavaTest.java > a8c58bd > typesystem/src/test/java/org/apache/atlas/typesystem/types/BaseTest.java > 1d80681 > > typesystem/src/test/java/org/apache/atlas/typesystem/types/TypeSystemTest.java > e8bbb0c > typesystem/src/test/resources/application.properties PRE-CREATION > > typesystem/src/test/scala/org/apache/atlas/typesystem/json/SerializationTest.scala > d14ac72 > > typesystem/src/test/scala/org/apache/atlas/typesystem/json/TypesSerializationTest.scala > ba4782c > webapp/pom.xml 6085d68 > webapp/src/main/java/org/apache/atlas/Main.java 7b13f3d > webapp/src/main/java/org/apache/atlas/examples/QuickStart.java b32b403 > webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java > bcf115f > webapp/src/main/java/org/apache/atlas/web/resources/EntitiesResource.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java > dcb460d > webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java > d6d5915 > webapp/src/main/java/org/apache/atlas/web/service/SecureEmbeddedServer.java > 1b2192c > webapp/src/main/resources/atlas-log4j.xml 7827c1a > > webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java > PRE-CREATION > > webapp/src/test/java/org/apache/atlas/web/listeners/TestGuiceServletConfig.java > 3ba6387 > webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java > 884d4de > > webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java > e6a7325 > > webapp/src/test/java/org/apache/atlas/web/resources/HiveLineageJerseyResourceIT.java > a884b5f > webapp/src/test/java/org/apache/atlas/web/security/BaseSecurityTest.java > 8af4a7e > > webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosIT.java > 0b95b7a > > webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosTest.java > PRE-CREATION > webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosIT.java > f00ac64 > webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosTest.java > PRE-CREATION > webapp/src/test/java/org/apache/atlas/web/security/SSLIT.java 3e23185 > webapp/src/test/java/org/apache/atlas/web/security/SSLTest.java > PRE-CREATION > > webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerIT.java > e1f9b54 > > webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerITBase.java > f7c3625 > > webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerTest.java > PRE-CREATION > > webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerTestBase.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/38393/diff/ > > > Testing > ------- > > UTs and ITs > > > Thanks, > > Shwetha GS > >
