-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51035/
-----------------------------------------------------------
Review request for atlas.
Bugs: ATLAS-1111
https://issues.apache.org/jira/browse/ATLAS-1111
Repository: atlas
Description
-------
kafkaconsumer.next() always returns the next message irrespective of auto
commit or not. With auto commit disabled, the the offset has to be committed
manually, as opposed to auto commit on next(). The last saved offset is used
across consumer restarts.
During graceful shutdown, because of graph shutdown, all the message processing
failed and we did commits even in case of failures. So, after restart, atlas
started at last committed offset and ignored message failures during shutdown
and hence lost messages.
The patch has the following changes:
1. No commit in case of message failure
2. Failed messages are written to different log file
Diffs
-----
distro/src/conf/atlas-log4j.xml eaa4ec5
notification/src/main/java/org/apache/atlas/kafka/KafkaConsumer.java 270215d
notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
2e861cb
notification/src/main/java/org/apache/atlas/notification/hook/HookNotification.java
88a0322
notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationMockTest.java
PRE-CREATION
notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java
2a49634
notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
0c8990f
plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
92cc406
typesystem/src/test/resources/atlas-application.properties a3b6c90
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
901b1ed
webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java
0a7c5df
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
6fd1939
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
d22c5f1
Diff: https://reviews.apache.org/r/51035/diff/
Testing
-------
UTs and end to end test
Thanks,
Shwetha GS
