Raymond created CAMEL-22168:
-------------------------------
Summary: PollEnrich with FTP ignores maxMessagesPerPoll
Key: CAMEL-22168
URL: https://issues.apache.org/jira/browse/CAMEL-22168
Project: Camel
Issue Type: Improvement
Components: camel-file, camel-ftp
Affects Versions: 4.12.0
Reporter: Raymond
The File and FTP components have the option {*}maxMessagesPerPoll{*}:
| |To define a maximum messages to gather per poll. By default no maximum is
set. Can be used to set a limit of e.g. 1000 to avoid when starting up the
server that there are thousands of files. Set a value of 0 or negative to
disabled it.|
However, when the FTP component (or any other camel-file based component) is
used in combination with the pollEnrich eip, this parameter is ignored.
{code:java}
<pollEnrich strategyRef="CurrentEnrichStrategy" timeout="5000">
<simple>sftp:myftpserver:2022/somepath?username=RAW(myuser)&password=RAW(mypassword)&include=RAW(.*.txt)&maxMessagesPerPoll=3</simple>
</pollEnrich>
{code}
This is well known, as the pollEnrich can only pick up one file per poll. A
solution can be found on StackOverflow:
[https://stackoverflow.com/questions/38103035/apache-camel-pollenrich-is-not-copying-all-the-files]
The problem is that you only can find out after running it. It's not clear that
the maxMessagesPerPoll isn't active/ignored (and this may as well count for
other parameters).
Besides, the maxMessagesPerPoll isn't working, even with the solution on
StackOverflow. To work around this, something like this could be done:
{code:java}
<route id="674f52f7-981e-4b3c-95a1-cbdf299b9ee9">
<from uri="direct:start"/>
<setProperty propertyName="Enrich-MaxMessagesPerPoll">
<constant>3</constant>
</setProperty>
<loop doWhile="true">
<simple>${exchangeProperty.Enrich-MaxMessagesPerPoll} != 0</simple>
<pollEnrich strategyRef="CurrentEnrichStrategy" timeout="5000">
<simple>sftp:myftpserver:2022/somepath?username=RAW(myuser)&password=RAW(mypassword)&include=RAW(.*.txt)&maxMessagesPerPoll=3</simple>
</pollEnrich>
<setProperty propertyName="Enrich-MaxMessagesPerPoll">
<simple>${exchangeProperty.Enrich-MaxMessagesPerPoll}--</simple>
</setProperty>
<choice>
<when>
<simple>${body} == null</simple>
<setProperty propertyName="Enrich-MaxMessagesPerPoll">
<constant>0</constant>
</setProperty>
</when>
<otherwise>
<to uri="direct:out"/>
<removeHeaders pattern="CamelFile*"/>
</otherwise>
</choice>
</loop>
</route> {code}
In the above example, -1 picks up all files, while a positive integer uses the
maxMessagesPerPoll. However, this solution is suboptimal, because every loop
needs to perform an FTP call (with large number this is resource intensive),
and uses lots of DSL code of something that should be simple.
I think at least a log warning should be printed when this combination is used
that this parameter is not active (and maybe this counts for other parameters
as well). The best thing would be if this would be possible (For example within
a pollEnrich, or a producer option on FTP/File to pickup files, or some other
solution).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
