[jira] [Resolved] (CAMEL-10333) (Consumer goes into loop while processing messages) DEBUG Cannot auto-commit offsets now since the coordinator is unknown

[Claus Ibsen (JIRA)]

     [ 
https://issues.apache.org/jira/browse/CAMEL-10333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Claus Ibsen resolved CAMEL-10333.
---------------------------------
    Resolution: Later

Try with latest release as camel-kafka has been improved/update

> (Consumer goes into loop while processing messages) DEBUG Cannot auto-commit 
> offsets now since the coordinator is unknown
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-10333
>                 URL: https://issues.apache.org/jira/browse/CAMEL-10333
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-kafka
>    Affects Versions: 2.17.1
>         Environment: Spring Framework is used for creating consumers, which 
> processes the messages.
>            Reporter: Shashank Madhav
>
> I have a topic which have 300 messages in the queue. When the camel is used 
> for processing the messages with event-based consumer, I am getting following 
> error after all the messages are processed.
> /////error message start
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            DEBUG Cannot 
> auto-commit offsets now since the coordinator is unknown, will retry after 
> backoff
> [ad #0 - KafkaTopic[Audit_logs]] AbstractCoordinator            DEBUG Issuing 
> group metadata request to broker 0
> [ad #0 - KafkaTopic[Audit_logs]] AbstractCoordinator            DEBUG Group 
> metadata response ClientResponse(receivedTimeMs=1473953020273, 
> disconnected=false, request=ClientRequest(expectResponse=true, 
> callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@1edc39d,
>  
> request=RequestSend(header={api_key=10,api_version=0,correlation_id=9,client_id=consumer-1},
>  body={group_id=test3}), createdTimeMs=1473953020218, 
> sendTimeMs=1473953020218), 
> responseBody={error_code=0,coordinator={node_id=0,host=<host-name>,port=<port-no>}})
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            ERROR Error 
> ILLEGAL_GENERATION occurred while committing offsets for group test3
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            WARN  Auto 
> offset commit failed: Commit cannot be completed due to group rebalance
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            ERROR Error 
> ILLEGAL_GENERATION occurred while committing offsets for group test3
> [ad #0 - KafkaTopic[Audit_logs]] ConsumerCoordinator            WARN  Auto 
> offset commit failed: 
> ////error message end
>  Following the error, it then starts from the beginning and processes the 
> entire queue again causing issues. The route used for camel is as follows 
> //camel-route start
>         <c:endpoint id="kafkaTest" 
> uri="kafka:${broker.server}?topic=${topics1}&amp;groupId=test3&amp;autoOffsetReset=earliest&amp;consumersCount=1&amp;sessionTimeoutMs=30000&amp;consumerRequestTimeoutMs=40000&amp;keyDeserializer=${broker.keyDeserializer}&amp;valueDeserializer=${broker.valueDeserializer}"/>
>               
>               <c:route id="route1">
>             <c:from uri="ref:kafkaTest"/>
>             <c:delay>
>             <c:constant>500</c:constant>
>             </c:delay>
>              <c:to uri="log:input"/> 
>         </c:route>
> //camel-route ends
> Delay is purposely added in the route to create message processing scenario. 
> If the delay is reduced or the sessionTimeoutMs is increased then it works 
> fine, but this is not a feasible solution. Without camel it works fine and 
> messages are processed properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)