> On July 13, 2017, 5:23 p.m., Sarath Subramanian wrote:
> > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java
> > Line 126 (original), 127 (patched)
> > <https://reviews.apache.org/r/60597/diff/1/?file=1768418#file1768418line127>
> >
> >     can the new kafka consumer handle both "smallest" and "earliest" value 
> > for auto.offset.reset?

the new kafka consumer can handle only earliest, smallest causes exception 
while initialization of kafkaconsumer. so set earliet as default which was set 
earlier.


> On July 13, 2017, 5:23 p.m., Sarath Subramanian wrote:
> > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java
> > Lines 133 (patched)
> > <https://reviews.apache.org/r/60597/diff/1/?file=1768418#file1768418line133>
> >
> >     should this be subsetConfiguration.getLong("session.timeout.ms", 3000)?

Tried your suggestion but there was a parse error (below). The kafka config is 
expecting the long value in string format

     272 org.apache.kafka.common.config.ConfigException: Invalid value 30000 
for configuration session.timeout.ms: Expected value to be an number.


> On July 13, 2017, 5:23 p.m., Sarath Subramanian wrote:
> > notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java
> > Line 102 (original), 102 (patched)
> > <https://reviews.apache.org/r/60597/diff/1/?file=1768420#file1768420line102>
> >
> >     why reduced poll timeout to 100L from 1000?

The return output of from poll method is mocked, so poll timeout will not 
matter as such, so set the timeout it less value to reduce execution time.


- Nixon


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60597/#review180445
-----------------------------------------------------------


On July 14, 2017, 9:52 a.m., Nixon Rodrigues wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60597/
> -----------------------------------------------------------
> 
> (Updated July 14, 2017, 9:52 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1908
>     https://issues.apache.org/jira/browse/ATLAS-1908
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> This patch handles old kafka consumer configs values to be added as new 
> configs if new configs are not present , so that Kafka consumer starts 
> gracefully even if new configs are not added explicitly.
> 
> eg. atlas.kafka.auto.commit.enable=false to 
> atlas.kafka.enable.auto.commit=false
>     atlas.kafka.auto.offset.reset=smallest to 
> atlas.kafka.auto.offset.reset=earliest
>     atlas.kafka.session.timeout.ms=30000
>     
>     Also added poll.timeout.ms in configs so kafka consumer can be 
> configurable.This involve change in AtlasKafkaConsumer receive interface.
> 
> 
> Diffs
> -----
> 
>   distro/src/conf/atlas-application.properties c3213df 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> 9c15243 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
> 366c8a7 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  22e40f9 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 
> 70059cb 
>   
> notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 
> c791d43 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  8324b57 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  0dea0e2 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
>  650ca0a 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> c036cfa 
> 
> 
> Diff: https://reviews.apache.org/r/60597/diff/2/
> 
> 
> Testing
> -------
> 
> Executed UT/IT's using mvn clean install.
> Tested KafkaConsumer with Hive Hook by creating table in hive.
> Atlas starts gracefully and Kafka consumer is up without error when new 
> properties are not added.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Reply via email to