-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38393/
-----------------------------------------------------------
Review request for atlas.
Bugs: ATLAS-58
https://issues.apache.org/jira/browse/ATLAS-58
Repository: atlas
Description
-------
Hive hook sends notification messages (list of entities). The Notification
consumer on server side consumes these messages and registers the entities. The
server handles de-duping of entities based on the unique attribute of the entity
Big changes:
1. Concept of service that are started started and stopped at atlas start and
stop
2. De-duping of entities on server based on any unique attribute for the
entity. If entity doesn't have any unique attribute, de-duping is not done and
new entity is created
3. Changed entity submit API to take list of entities instead of just 1 entity
(required for hive hook)
4. Moved security tests from integration tests to unit tests - as they were
creating issues with server start as jetty already starts another server for
integration tests
5. Removed some duplicate tests from repository module (the same tests exist in
typesystem module as well)
6. In webapp ITs, re-used the types defined
7. Hive hook now sends notifications instead of registering entities. Sending
notification is done synchronously. So, this adds to hive command execution
delay. But this also makes it reliable
Pending:
1. Entity updates like alter table commands are not handlded. Will create
another jira for this
2. Webapp jetty plugin doesn't shutdown embedded kafka at the end of
integration tests. So, hive bridge ITs fail. Hive bridge ITs pass if run on
their own. Still checking on this
Diffs
-----
addons/hive-bridge/pom.xml 914d8c6
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
4d1af4e
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
fe26446
addons/hive-bridge/src/main/java/org/apache/atlas/hive/model/HiveDataModelGenerator.java
cb8359e
addons/hive-bridge/src/site/twiki/Bridge-Hive.twiki 647af27
addons/hive-bridge/src/test/java/org/apache/atlas/hive/hook/HiveHookIT.java
2bf3aba
client/pom.xml 2e27930
client/src/main/java/org/apache/atlas/ApplicationProperties.java 738ec53
client/src/main/java/org/apache/atlas/AtlasClient.java 1c7d62a
client/src/test/java/org/apache/atlas/ApplicationPropertiesTest.java 4acb3e1
client/src/test/resources/application.properties dbd6002
common/pom.xml PRE-CREATION
common/src/main/java/org/apache/atlas/service/Service.java PRE-CREATION
common/src/main/java/org/apache/atlas/service/Services.java PRE-CREATION
dashboard/public/modules/details/detailsResource.js af2f249
distro/pom.xml d42d78f
distro/src/conf/application.properties bf323a7
docs/src/site/twiki/InstallationSteps.twiki 3d0351c
notification/pom.xml b036855
notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java
9978275
notification/src/main/java/org/apache/atlas/kafka/KafkaNotificationProvider.java
PRE-CREATION
notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
36a62f0
notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
0951124
notification/src/main/java/org/apache/atlas/notification/NotificationModule.java
db17e35
notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java
02752dc
pom.xml e8d18a2
repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java
3d47293
repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java
fbd01de
repository/src/main/java/org/apache/atlas/listener/EntityChangeListener.java
f58d6de
repository/src/main/java/org/apache/atlas/repository/EntityExistsException.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java
1b0faa5
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
cc10ea3
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
ceb8a65
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
56168db
repository/src/main/java/org/apache/atlas/services/MetadataService.java
8f82f88
repository/src/test/java/org/apache/atlas/RepositoryServiceLoadingTest.java
18b1d1b
repository/src/test/java/org/apache/atlas/TestUtils.java de41793
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
127e6d7
repository/src/test/java/org/apache/atlas/discovery/HiveLineageServiceTest.java
2c7d61a
repository/src/test/java/org/apache/atlas/repository/BaseTest.java 8924c37
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
487ac84
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java
be99efe
repository/src/test/java/org/apache/atlas/repository/memory/ClassTest.java
ed747b2
repository/src/test/java/org/apache/atlas/repository/memory/EnumTest.java
4ce1085
repository/src/test/java/org/apache/atlas/repository/memory/InstanceE2ETest.java
3c29bf8
repository/src/test/java/org/apache/atlas/repository/memory/StorageTest.java
c329caf
repository/src/test/java/org/apache/atlas/repository/memory/StructTest.java
3ab7020
repository/src/test/java/org/apache/atlas/repository/memory/TraitTest.java
bfdc8b1
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
PRE-CREATION
repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala
eef13df
repository/src/test/scala/org/apache/atlas/query/QueryTestsUtils.scala
ec21886
typesystem/pom.xml e4a1383
typesystem/src/main/java/org/apache/atlas/ApplicationProperties.java
PRE-CREATION
typesystem/src/main/java/org/apache/atlas/TypeExistsException.java
PRE-CREATION
typesystem/src/main/java/org/apache/atlas/typesystem/persistence/StructInstance.java
309ab11
typesystem/src/main/java/org/apache/atlas/typesystem/types/AttributeDefinition.java
31b5f47
typesystem/src/main/java/org/apache/atlas/typesystem/types/AttributeInfo.java
e74f8d5
typesystem/src/main/java/org/apache/atlas/typesystem/types/ObjectGraphWalker.java
399902c
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java
d1e8dbd
typesystem/src/main/java/org/apache/atlas/typesystem/types/ValueConversionException.java
5cdee8d
typesystem/src/main/resources/application.properties f7e2774
typesystem/src/main/resources/atlas-log4j.xml PRE-CREATION
typesystem/src/main/scala/org/apache/atlas/typesystem/json/TypesSerialization.scala
7a681bf
typesystem/src/test/java/org/apache/atlas/ApplicationPropertiesTest.java
PRE-CREATION
typesystem/src/test/java/org/apache/atlas/typesystem/json/SerializationJavaTest.java
a8c58bd
typesystem/src/test/java/org/apache/atlas/typesystem/types/BaseTest.java
1d80681
typesystem/src/test/java/org/apache/atlas/typesystem/types/TypeSystemTest.java
e8bbb0c
typesystem/src/test/resources/application.properties PRE-CREATION
typesystem/src/test/scala/org/apache/atlas/typesystem/json/SerializationTest.scala
d14ac72
typesystem/src/test/scala/org/apache/atlas/typesystem/json/TypesSerializationTest.scala
ba4782c
webapp/pom.xml 6085d68
webapp/src/main/java/org/apache/atlas/Main.java 7b13f3d
webapp/src/main/java/org/apache/atlas/examples/QuickStart.java b32b403
webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java
bcf115f
webapp/src/main/java/org/apache/atlas/web/resources/EntitiesResource.java
PRE-CREATION
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java
dcb460d
webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java
d6d5915
webapp/src/main/java/org/apache/atlas/web/service/SecureEmbeddedServer.java
1b2192c
webapp/src/main/resources/atlas-log4j.xml 7827c1a
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/listeners/TestGuiceServletConfig.java
3ba6387
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java
884d4de
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java
e6a7325
webapp/src/test/java/org/apache/atlas/web/resources/HiveLineageJerseyResourceIT.java
a884b5f
webapp/src/test/java/org/apache/atlas/web/security/BaseSecurityTest.java
8af4a7e
webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosIT.java
0b95b7a
webapp/src/test/java/org/apache/atlas/web/security/NegativeSSLAndKerberosTest.java
PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosIT.java
f00ac64
webapp/src/test/java/org/apache/atlas/web/security/SSLAndKerberosTest.java
PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/security/SSLIT.java 3e23185
webapp/src/test/java/org/apache/atlas/web/security/SSLTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerIT.java
e1f9b54
webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerITBase.java
f7c3625
webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerTest.java
PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/service/SecureEmbeddedServerTestBase.java
PRE-CREATION
Diff: https://reviews.apache.org/r/38393/diff/
Testing
-------
UTs and ITs
Thanks,
Shwetha GS
