[
https://issues.apache.org/jira/browse/NIFI-13738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Will James updated NIFI-13738:
------------------------------
Description:
Currently if the ConsumeAMQP attempts to consume a message larger than 64MiB
from the broker (tested with RabbitMQ), the following error appears on the
processor's bulletin:
{code:java}
java.lang.IllegalStateException: Message body is too large (132120576), maximum
configured size is 67108864. See ConnectionFactory#setMaxInboundMessageBodySize
if you need to increase the limit{code}
Followed by a `connection lost` error.
The message remains unconsumed in the broker and the ConsumeAMQP processor will
not be able to consume anything until the problem message is cleared.
After some initial investigation, it looks like the rabbitmq.client library has
been updated in the in the last year to add a setMaxInboundMessageBodySize
property to Consuming connections (with a default max of 64MiB). Unfortunately,
this property has not be configured/implemented in the NiFi ConsumeAMQP
processor, causing the processor to block anything larger than 64MiB.
Ideally, the setMaxInboundMessageBodySize property would be configurable as a
NiFi UI property.
Changes will need to be made to the AMQP processors in
/{_}nifi-extension-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/{_}
was:
Currently if the ConsumeAMQP attempts to consume a message larger than 64MiB
from the broker (tested with RabbitMQ), the following error appears on the
processor's bulletin:
java.lang.IllegalStateException: Message body is too large (132120576), maximum
configured size is 67108864. See ConnectionFactory#setMaxInboundMessageBodySize
if you need to increase the limit
Followed by a `connection lost` error.
The message remains unconsumed in the broker and the ConsumeAMQP processor will
not be able to consume anything until the problem message is cleared.
After some initial investigation, it looks like the rabbitmq.client library has
been updated in the in the last year to add a setMaxInboundMessageBodySize
property to Consuming connections (with a default max of 64MiB). Unfortunately,
this property has not be configured/implemented in the NiFi ConsumeAMQP
processor, causing the processor to block anything larger than 64MiB.
Ideally, the setMaxInboundMessageBodySize property would be configurable as a
NiFi UI property.
Changes will need to be made to the AMQP processors in
/{_}nifi-extension-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/{_}
> ConsumeAMQP processor won't consume messages greater than 64MiB
> ---------------------------------------------------------------
>
> Key: NIFI-13738
> URL: https://issues.apache.org/jira/browse/NIFI-13738
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.26.0, 1.27.0
> Environment: Tested on RHEL Linux when connected to RabbitMQ
> Reporter: Will James
> Priority: Major
>
> Currently if the ConsumeAMQP attempts to consume a message larger than 64MiB
> from the broker (tested with RabbitMQ), the following error appears on the
> processor's bulletin:
> {code:java}
> java.lang.IllegalStateException: Message body is too large (132120576),
> maximum configured size is 67108864. See
> ConnectionFactory#setMaxInboundMessageBodySize if you need to increase the
> limit{code}
> Followed by a `connection lost` error.
> The message remains unconsumed in the broker and the ConsumeAMQP processor
> will not be able to consume anything until the problem message is cleared.
> After some initial investigation, it looks like the rabbitmq.client library
> has been updated in the in the last year to add a
> setMaxInboundMessageBodySize property to Consuming connections (with a
> default max of 64MiB). Unfortunately, this property has not be
> configured/implemented in the NiFi ConsumeAMQP processor, causing the
> processor to block anything larger than 64MiB.
> Ideally, the setMaxInboundMessageBodySize property would be configurable as a
> NiFi UI property.
> Changes will need to be made to the AMQP processors in
> /{_}nifi-extension-bundles/nifi-amqp-bundle/nifi-amqp-processors/src/main/java/org/apache/nifi/amqp/processors/{_}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
