[jira] [Commented] (KAFKA-3737) Closing connection during produce request should be log with WARN level.

[Florian Hussonnois (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329482#comment-15329482
 ] 

Florian Hussonnois commented on KAFKA-3737:
---

Hi Gwen, 

My concern is more about the kafka.common.MessageSizeTooLargeException which is 
logged with the INFO level. 

Actually, I think most of users are collecting error messages based on log 
level in order to detect anomalies in a production environment.
I don't know if property "max.message.bytes" is controlled on client side or on 
broker side.
If this property is controlled on broker side it could be more convenient to 
control it before sending from client ?

This would allow to not overload brokers in case of too heavy messages. 
Moreover, this could improve security producers will not be allowed to send 
very big messages.

Thanks,

> Closing connection during produce request should be log with WARN level.
> 
>
> Key: KAFKA-3737
> URL: https://issues.apache.org/jira/browse/KAFKA-3737
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 0.9.0.1
>Reporter: Florian Hussonnois
>Priority: Trivial
>
> Currently if an an error occurred during a produce request the exeception is 
> log as info.
> INFO [KafkaApi-0] Closing connection due to error during produce request with 
> correlation id 24 from client id console-producer with ack=0
> Topic and partition to exceptions: [test,0] -> 
> kafka.common.MessageSizeTooLargeException (kafka.server.KafkaApis)
> It could be more conveniant to use a WARN level to ease the tracing of this 
> errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3737) Closing connection during produce request should be log with WARN level.

[Gwen Shapira (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299191#comment-15299191
 ] 

Gwen Shapira commented on KAFKA-3737:
-

I don't mind bumping up the LOG level, but I don't think this is the log you 
were looking for...

This INFO is only when data is sent with "request.required.acks=0" from the 
consumer, which theoretically should mean that the producer doesn't care 
whether the data made it or not (by the time we write this log, the consumer 
already assumes everything is amazing, which is why we need to close the 
connector to get its attention).

Maybe the log message you are looking for is:
" debug("Produce request with correlation id %d from client %s on partition %s 
failed due to %s".format("
Few lines up? 

This is for all the produce errors, but then, this can flood the logs pretty 
easily. Brokers handle thousands produce requests per second...



In general:
1. The best way to detect producer errors is from the producer log... 
2. We recommend running the broker log at INFO level anyway :)

> Closing connection during produce request should be log with WARN level.
> 
>
> Key: KAFKA-3737
> URL: https://issues.apache.org/jira/browse/KAFKA-3737
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 0.9.0.1
>Reporter: Florian Hussonnois
>Priority: Trivial
>
> Currently if an an error occurred during a produce request the exeception is 
> log as info.
> INFO [KafkaApi-0] Closing connection due to error during produce request with 
> correlation id 24 from client id console-producer with ack=0
> Topic and partition to exceptions: [test,0] -> 
> kafka.common.MessageSizeTooLargeException (kafka.server.KafkaApis)
> It could be more conveniant to use a WARN level to ease the tracing of this 
> errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3737) Closing connection during produce request should be log with WARN level.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293440#comment-15293440
 ] 

ASF GitHub Bot commented on KAFKA-3737:
---

GitHub user fhussonnois opened a pull request:

https://github.com/apache/kafka/pull/1415

KAFKA-3737: Change log level for error during produce request

Minor change for https://issues.apache.org/jira/browse/KAFKA-3737

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/fhussonnois/kafka kafka-3737

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/1415.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1415


commit e2dac4a834c84808a8d171dbfdc149f6207e6d4a
Author: Florian Hussonnois 
Date:   2016-05-20T14:24:29Z

KAFKA-3737: Change log level for error during produce request




> Closing connection during produce request should be log with WARN level.
> 
>
> Key: KAFKA-3737
> URL: https://issues.apache.org/jira/browse/KAFKA-3737
> Project: Kafka
>  Issue Type: Bug
>  Components: core
>Affects Versions: 0.9.0.1
>Reporter: Florian Hussonnois
>Priority: Trivial
>
> Currently if an an error occurred during a produce request the exeception is 
> log as info.
> INFO [KafkaApi-0] Closing connection due to error during produce request with 
> correlation id 24 from client id console-producer with ack=0
> Topic and partition to exceptions: [test,0] -> 
> kafka.common.MessageSizeTooLargeException (kafka.server.KafkaApis)
> It could be more conveniant to use a WARN level to ease the tracing of this 
> errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)