Question about Camel Connector

2019-03-06 Thread Willem Jiang
Hi,

I think Camel Connector[1] is quit useful if we have some
configuration need to be done on the component or endpoint level.
I just found the codes of CamelConnector were removed in the camel
3.x.  But there is not much discussion about it. May I know the reason
why do we deprecate these connectors and remove them in Camel 3.x?

Thanks,

[1]https://issues.apache.org/jira/browse/CAMEL-10721

Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem


Re: Automatic documentation of a component

2019-03-06 Thread Claus Ibsen
Hi

No it should not, but its the update-readme goal of the
camel-package-maven-plugin that does this.

Mind that this package tool is written with mind of helping with the
Camel project itself, and not specific to Camel end users.
So there can be some corner cases where you can't do the same.


On Wed, Mar 6, 2019 at 6:11 PM Stefan Ziegler
 wrote:
>
> Does the code has to be in the "org.apache.camel.component" package? Or can
> it be anything?
>
> Stefan
>
> On Wed, Mar 6, 2019 at 3:52 PM Claus Ibsen  wrote:
>
> > Hi
> >
> > Yeah the adoc file should be in src/main/docs and be named
> >
> > componentName-component.adoc
> >
> > so if the component is named acme, then it should be
> >
> > acme-component.adoc
> >
> >
> > On Wed, Mar 6, 2019 at 2:58 PM Stefan Ziegler
> >  wrote:
> > >
> > > Hi
> > >
> > > I used the maven archetype so I think it should be all there.
> > >
> > > Which names have to match? The name of the .adoc file must be identical
> > to
> > > ...?
> > >
> > > regards
> > > Stefan
> > >
> > > On Wed, Mar 6, 2019 at 12:48 PM Claus Ibsen 
> > wrote:
> > >
> > > > Hi
> > > >
> > > > You need to have a maven plugin added and another dependency.
> > > >
> > > > See this pom
> > > >
> > > >
> > https://github.com/apache/camel/blob/camel-2.x/connectors/examples/beverage-component/pom.xml
> > > >
> > > > Also the maven archetype for component should include all of this also.
> > > >
> > > > On Wed, Mar 6, 2019 at 12:07 PM Stefan Ziegler
> > > >  wrote:
> > > > >
> > > > > Hi
> > > > >
> > > > > I'm writing my first apache camel component:
> > > > > https://github.com/edigonzales/camel-ili2pg
> > > > >
> > > > > I'm wondering how to get the automatic documentation of the
> > component and
> > > > > endpoint options in the asciidoc file? I have put an emtpy .adoc
> > file in
> > > > > src/main/docs/ only with
> > > > >
> > > > > // component options: START
> > > > > // component options: END
> > > > >
> > > > > // endpoint options: START
> > > > > // endpoint options: END
> > > > >
> > > > > written in it.
> > > > >
> > > > > I then run "mvn clean install" but I couldn't find any updated / new
> > > > > asciidoc file with the component options. Endpoints etc are
> > annotated.
> > > > >
> > > > > Thanks for any hints.
> > > > > regards
> > > > > Stefan
> > > >
> > > >
> > > >
> > > > --
> > > > Claus Ibsen
> > > > -
> > > > http://davsclaus.com @davsclaus
> > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > >
> >
> >
> >
> > --
> > Claus Ibsen
> > -
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2


Re: Automatic documentation of a component

2019-03-06 Thread Stefan Ziegler
Does the code has to be in the "org.apache.camel.component" package? Or can
it be anything?

Stefan

On Wed, Mar 6, 2019 at 3:52 PM Claus Ibsen  wrote:

> Hi
>
> Yeah the adoc file should be in src/main/docs and be named
>
> componentName-component.adoc
>
> so if the component is named acme, then it should be
>
> acme-component.adoc
>
>
> On Wed, Mar 6, 2019 at 2:58 PM Stefan Ziegler
>  wrote:
> >
> > Hi
> >
> > I used the maven archetype so I think it should be all there.
> >
> > Which names have to match? The name of the .adoc file must be identical
> to
> > ...?
> >
> > regards
> > Stefan
> >
> > On Wed, Mar 6, 2019 at 12:48 PM Claus Ibsen 
> wrote:
> >
> > > Hi
> > >
> > > You need to have a maven plugin added and another dependency.
> > >
> > > See this pom
> > >
> > >
> https://github.com/apache/camel/blob/camel-2.x/connectors/examples/beverage-component/pom.xml
> > >
> > > Also the maven archetype for component should include all of this also.
> > >
> > > On Wed, Mar 6, 2019 at 12:07 PM Stefan Ziegler
> > >  wrote:
> > > >
> > > > Hi
> > > >
> > > > I'm writing my first apache camel component:
> > > > https://github.com/edigonzales/camel-ili2pg
> > > >
> > > > I'm wondering how to get the automatic documentation of the
> component and
> > > > endpoint options in the asciidoc file? I have put an emtpy .adoc
> file in
> > > > src/main/docs/ only with
> > > >
> > > > // component options: START
> > > > // component options: END
> > > >
> > > > // endpoint options: START
> > > > // endpoint options: END
> > > >
> > > > written in it.
> > > >
> > > > I then run "mvn clean install" but I couldn't find any updated / new
> > > > asciidoc file with the component options. Endpoints etc are
> annotated.
> > > >
> > > > Thanks for any hints.
> > > > regards
> > > > Stefan
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>


RE: DeadLetterChannel handle option

2019-03-06 Thread Valdis Andersons
Hey,

That's pretty much how I would understand it was well. Though I'm not setting 
the handled option explicitly anywhere in my route builders. To make the 
exchange go somewhere else in case of an exception you'd really just have to 
specify the deadLetterChannel in the errorHandler of your route builder:

errorHandler(deadLetterChannel(deadLetterQueue));

Optionally you can also define a retry strategy on some exceptions while not on 
others:

onException(RecoverableException.class)

.maximumRedeliveries(redeliverable.maximumRedeliveries())

.redeliveryDelay(redeliverable.redeliveryDelay())
.logExhaustedMessageHistory(true)

.retryAttemptedLogLevel(LoggingLevel.WARN)
.useOriginalMessage();

Or whatever fits your use case and requirements.

To get the log output for an exception that isn't specified for the retry (in 
the onException configuration) I had to configure my logging properties this 
way:

logging:
level:
org:
apache:
camel: ERROR
processor:
DeadLetterChannel: DEBUG

That will print a line in the logs every time something gets dead-lettered due 
to an exception that hasn't been handled in the code.


Cheers,
Valdis

-Original Message-
From: Wang Yan [mailto:wyan...@gmail.com] 
Sent: 06 March 2019 13:17
To: users@camel.apache.org
Subject: DeadLetterChannel handle option

Hello All,

based on below document
if handle =false the message will be propagated to caller when exception 
happened if handle=true, the message will not be propagated to caller but 
somewhere else as you can define . is my understanding correct?

http://scanmail.trustwave.com/?c=6600=2sj_3M0KVwaA2voGn_01oHSf2lEMV672cpA88z2Sww=33=http%3a%2f%2fcamel%2eapache%2eorg%2fdead-letter-channel%2ehtml
The *DeadLetterChannel* lets you control behaviors including redelivery, 
whether to propagate the thrown Exception to the caller (the *handled* option), 
and where the (failed) Exchange should now be routed to.

Vhi Group DAC (Vhi) is a holding company for insurance and healthcare services, 
which include Vhi Healthcare DAC, Vhi Insurance DAC, Vhi Health Services DAC 
and Vhi Investments DAC. Vhi Healthcare DAC trading as Vhi Healthcare and Vhi 
Insurance DAC trading as Vhi Insurance are regulated by the Central Bank of 
Ireland. Vhi Healthcare is tied to Vhi Insurance DAC for health insurance in 
Ireland which is underwritten by Vhi Insurance DAC. Vhi Healthcare is tied to 
Zurich Life Assurance plc for Vhi Life Term Insurance and Vhi Mortgage 
Protection which is underwritten by Zurich Life Assurance plc. Vhi Healthcare 
is tied to Collinson Insurance Services Limited for MultiTrip Travel Insurance, 
Backpacker Travel Insurance and Vhi Dental Insurance which are underwritten by 
Great Lakes Insurance SE, UK branch and for Vhi Canada Cover and Vhi 
International Health Insurance which are underwritten by Astrenska Insurance 
Limited. For more information about the Vhi Group please go to: 
https://www.vhi.ie/about-vhi. 


Tá Vhi Group DAC (Vhi) ina chuideachta sealbhaíochta le haghaidh seirbhísí 
árachais agus seirbhísí cúram sláinte, lena n-áirítear Vhi Healthcare DAC, Vhi 
Insurance DAC, Vhi Health Services DAC agus Vhi Investments DAC. Déanann Banc 
Ceannais na hÉireann rialáil ar Vhi Healthcare DAC, ag trádáil dó mar Vhi 
Healthcare, agus ar Vhi Insurance DAC, ag trádáil dó mar Vhi Insurance. Tá Vhi 
Healthcare ceangailte le Vhi Insurance DAC le haghaidh árachas sláinte in 
Éirinn, rud atá frithgheallta ag Vhi Insurance DAC. Tá Vhi Healthcare 
ceangailte le Zurich Life Assurance plc le haghaidh Árachais Saoil de chuid Vhi 
agus Árachas Cosanta Morgáiste de chuid Vhi atá frithgheallta ag Zurich Life 
Assurance plc. Tá Vhi Healthcare ceangailte le Collinson Insurance Services 
Limited le haghaidh Árachas Taistil Ilturais agus Turasóirí Mála Droma agus 
Árachas Fiaclóireachta de chuid Vhi atá frithgheallta ag Great Lakes Insurance 
SE, UK branch agus le haghaidh Clúdach Cheanada de chuid Vhi agus Árachas 
Sláinte Idirnáisiúnta de chuid Vhi atá frithgheallta ag Astrenska Insurance 
Limited. Chun tuilleadh faisnéise a fháil faoi Ghrúpa Vhi, tabhair cuairt ar: 
https://www.vhi.ie/about-vhi. 

This e-mail and any files transmitted with it contain information which may be 
confidential and which may also be privileged and is intended solely for the 
use of the individual or entity to whom it is addressed. Unless you are the 
intended recipient you may not copy or use it, or disclose it to anyone else. 
Any opinions expressed are that of the individual and not necessarily that of 
the Vhi Group. If you have received this e-mail in error please notify the 
sender by return.








Re: Automatic documentation of a component

2019-03-06 Thread Claus Ibsen
Hi

Yeah the adoc file should be in src/main/docs and be named

componentName-component.adoc

so if the component is named acme, then it should be

acme-component.adoc


On Wed, Mar 6, 2019 at 2:58 PM Stefan Ziegler
 wrote:
>
> Hi
>
> I used the maven archetype so I think it should be all there.
>
> Which names have to match? The name of the .adoc file must be identical to
> ...?
>
> regards
> Stefan
>
> On Wed, Mar 6, 2019 at 12:48 PM Claus Ibsen  wrote:
>
> > Hi
> >
> > You need to have a maven plugin added and another dependency.
> >
> > See this pom
> >
> > https://github.com/apache/camel/blob/camel-2.x/connectors/examples/beverage-component/pom.xml
> >
> > Also the maven archetype for component should include all of this also.
> >
> > On Wed, Mar 6, 2019 at 12:07 PM Stefan Ziegler
> >  wrote:
> > >
> > > Hi
> > >
> > > I'm writing my first apache camel component:
> > > https://github.com/edigonzales/camel-ili2pg
> > >
> > > I'm wondering how to get the automatic documentation of the component and
> > > endpoint options in the asciidoc file? I have put an emtpy .adoc file in
> > > src/main/docs/ only with
> > >
> > > // component options: START
> > > // component options: END
> > >
> > > // endpoint options: START
> > > // endpoint options: END
> > >
> > > written in it.
> > >
> > > I then run "mvn clean install" but I couldn't find any updated / new
> > > asciidoc file with the component options. Endpoints etc are annotated.
> > >
> > > Thanks for any hints.
> > > regards
> > > Stefan
> >
> >
> >
> > --
> > Claus Ibsen
> > -
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2


4 Apache Events in 2019: DC Roadshow soon; next up Chicago, Las Vegas, and Berlin!

2019-03-06 Thread Rich Bowen
Dear Apache Enthusiast,

(You’re receiving this because you are subscribed to one or more user
mailing lists for an Apache Software Foundation project.)

TL;DR:
 * Apache Roadshow DC is in 3 weeks. Register now at
https://apachecon.com/usroadshowdc19/
 * Registration for Apache Roadshow Chicago is open.
http://apachecon.com/chiroadshow19
 * The CFP for ApacheCon North America is now open.
https://apachecon.com/acna19
 * Save the date: ApacheCon Europe will be held in Berlin, October 22nd
through 24th.  https://apachecon.com/aceu19


Registration is open for two Apache Roadshows; these are smaller events
with a more focused program and regional community engagement:

Our Roadshow event in Washington DC takes place in under three weeks, on
March 25th. We’ll be hosting a day-long event at the Fairfax campus of
George Mason University. The roadshow is a full day of technical talks
(two tracks) and an open source job fair featuring AWS, Bloomberg, dito,
GridGain, Linode, and Security University. More details about the
program, the job fair, and to register, visit
https://apachecon.com/usroadshowdc19/

Apache Roadshow Chicago will be held May 13-14th at a number of venues
in Chicago’s Logan Square neighborhood. This event will feature sessions
in AdTech, FinTech and Insurance, startups, “Made in Chicago”, Project
Shark Tank (innovations from the Apache Incubator), community diversity,
and more. It’s a great way to learn about various Apache projects “at
work” while playing at a brewery, a beercade, and a neighborhood bar.
Sign up today at https://www.apachecon.com/chiroadshow19/

We’re delighted to announce that the Call for Presentations (CFP) is now
open for ApacheCon North America in Las Vegas, September 9-13th! As the
official conference series of the ASF, ApacheCon North America will
feature over a dozen Apache project summits, including Cassandra,
Cloudstack, Tomcat, Traffic Control, and more. We’re looking for talks
in a wide variety of categories -- anything related to ASF projects and
the Apache development process. The CFP closes at midnight on May 26th.
In addition, the ASF will be celebrating its 20th Anniversary during the
event. For more details and to submit a proposal for the CFP, visit
https://apachecon.com/acna19/ . Registration will be opening soon.

Be sure to mark your calendars for ApacheCon Europe, which will be held
in Berlin, October 22-24th at the KulturBrauerei, a landmark of Berlin's
industrial history. In addition to innovative content from our projects,
we are collaborating with the Open Source Design community
(https://opensourcedesign.net/) to offer a track on design this year.
The CFP and registration will open soon at https://apachecon.com/aceu19/ .

Sponsorship opportunities are available for all events, with details
listed on each event’s site at http://apachecon.com/.

We look forward to seeing you!

Rich, for the ApacheCon Planners
@apachecon



Re: Automatic documentation of a component

2019-03-06 Thread Stefan Ziegler
Hi

I used the maven archetype so I think it should be all there.

Which names have to match? The name of the .adoc file must be identical to
...?

regards
Stefan

On Wed, Mar 6, 2019 at 12:48 PM Claus Ibsen  wrote:

> Hi
>
> You need to have a maven plugin added and another dependency.
>
> See this pom
>
> https://github.com/apache/camel/blob/camel-2.x/connectors/examples/beverage-component/pom.xml
>
> Also the maven archetype for component should include all of this also.
>
> On Wed, Mar 6, 2019 at 12:07 PM Stefan Ziegler
>  wrote:
> >
> > Hi
> >
> > I'm writing my first apache camel component:
> > https://github.com/edigonzales/camel-ili2pg
> >
> > I'm wondering how to get the automatic documentation of the component and
> > endpoint options in the asciidoc file? I have put an emtpy .adoc file in
> > src/main/docs/ only with
> >
> > // component options: START
> > // component options: END
> >
> > // endpoint options: START
> > // endpoint options: END
> >
> > written in it.
> >
> > I then run "mvn clean install" but I couldn't find any updated / new
> > asciidoc file with the component options. Endpoints etc are annotated.
> >
> > Thanks for any hints.
> > regards
> > Stefan
>
>
>
> --
> Claus Ibsen
> -
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>


DeadLetterChannel handle option

2019-03-06 Thread Wang Yan
Hello All,

based on below document
if handle =false the message will be propagated to caller when exception
happened
if handle=true, the message will not be propagated to caller but somewhere
else as you can
define . is my understanding correct?

http://camel.apache.org/dead-letter-channel.html
The *DeadLetterChannel* lets you control behaviors including redelivery,
whether to propagate the thrown Exception to the caller (the *handled* option),
and where the (failed) Exchange should now be routed to.


Re: Automatic documentation of a component

2019-03-06 Thread Claus Ibsen
Hi

You need to have a maven plugin added and another dependency.

See this pom
https://github.com/apache/camel/blob/camel-2.x/connectors/examples/beverage-component/pom.xml

Also the maven archetype for component should include all of this also.

On Wed, Mar 6, 2019 at 12:07 PM Stefan Ziegler
 wrote:
>
> Hi
>
> I'm writing my first apache camel component:
> https://github.com/edigonzales/camel-ili2pg
>
> I'm wondering how to get the automatic documentation of the component and
> endpoint options in the asciidoc file? I have put an emtpy .adoc file in
> src/main/docs/ only with
>
> // component options: START
> // component options: END
>
> // endpoint options: START
> // endpoint options: END
>
> written in it.
>
> I then run "mvn clean install" but I couldn't find any updated / new
> asciidoc file with the component options. Endpoints etc are annotated.
>
> Thanks for any hints.
> regards
> Stefan



-- 
Claus Ibsen
-
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2


Automatic documentation of a component

2019-03-06 Thread Stefan Ziegler
Hi

I'm writing my first apache camel component:
https://github.com/edigonzales/camel-ili2pg

I'm wondering how to get the automatic documentation of the component and
endpoint options in the asciidoc file? I have put an emtpy .adoc file in
src/main/docs/ only with

// component options: START
// component options: END

// endpoint options: START
// endpoint options: END

written in it.

I then run "mvn clean install" but I couldn't find any updated / new
asciidoc file with the component options. Endpoints etc are annotated.

Thanks for any hints.
regards
Stefan


Re: imaps component not retrieving specified number of messages

2019-03-06 Thread Claus Ibsen
Hi

Yeah we can likely improve this to avoid fetching all the mails. You
are welcome to log a JIRA and work on a patch as github PR.
http://camel.apache.org/support.html
https://github.com/apache/camel/blob/master/CONTRIBUTING.md

On Wed, Mar 6, 2019 at 9:54 AM js504  wrote:
>
> Hello,
>
> I am trying to poll a gmail account to process emails using the camel-mail 
> version 2.23.0 imaps component and the Java DSL.
>
> There are thousands of emails I need to process but I am trying to split it 
> up so as to not overload anything.
>
> I have tried setting maxMessagesPerPoll and fetchSize parameters to different 
> numbers but no matter what it seems to try to peek every message first ( I 
> think ) and doesn't seem to retrieve just the number of messages I have 
> specified.  I have tried setting peek to false as well but this does not seem 
> to help.  I have also tried setting just the maxMessagesPerPoll without 
> fetchSize and peek as well as setting just fetchSize without 
> maxMessagesPerPoll or peek set but the same occurs.
>
> below is my from statement using the imaps component:
>
> from("imaps://{{env:IMAPS_HOSTNAME}}:{{env:IMAPS_PORT}}?" +
> "maxMessagesPerPoll=10" +
> "=10" +
> "=false" +
> "={{env:EMAIL_ADDRESS}}" +
> "=" + password +
> "={{env:IMAP_DEBUG_MODE}}" +
> "=5000")
>
> The following is the output from the connection to the gmail account and the 
> following '(BODY.PEEK[HEADER.FIELDS (Message-ID)])'  messages that seem to go 
> on and on for every message before retrieval of the message body has even 
> begun.
> Any explanation as to what is happening and what I could do in order to 
> process say 1000 emails at a time and stop it from doing this pre-processing 
> on all emails instead of the first 1000 per poll would be really appreciated.
>
> 2019-03-05 21:10:01,531  INFO [main] (DefaultCamelContext.java:3197) - Apache 
> Camel 2.23.1 (CamelContext: consume-emails) started in 0.969 seconds
> DEBUG: getProvider() returning 
> javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle]
> DEBUG IMAPS: mail.imap.fetchsize: 16384
> DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
> DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
> DEBUG IMAPS: mail.imap.appendbuffersize: -1
> DEBUG IMAPS: mail.imap.minidletime: 10
> DEBUG IMAPS: closeFoldersOnStoreFailure
> DEBUG IMAPS: trying to connect to host "imap.gmail.com", port 993, isSSL true
> * OK Gimap ready for requests from 45.2.196.68 n3mb457716515jap
> A0 CAPABILITY
> * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN 
> X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN 
> AUTH=OAUTHBEARER AUTH=XOAUTH
> A0 OK Thats all she wrote! n3mb457716515jap
> DEBUG IMAPS: AUTH: XOAUTH2
> DEBUG IMAPS: AUTH: PLAIN
> DEBUG IMAPS: AUTH: PLAIN-CLIENTTOKEN
> DEBUG IMAPS: AUTH: OAUTHBEARER
> DEBUG IMAPS: AUTH: XOAUTH
> DEBUG IMAPS: protocolConnect login, host=imap.gmail.com, user= me>, password=
> DEBUG IMAPS: AUTHENTICATE PLAIN command trace suppressed
> DEBUG IMAPS: AUTHENTICATE PLAIN command result: A1 OK  
> authenticated (Success)
> A2 ENABLE UTF8=ACCEPT
> * ENABLED UTF8=ACCEPT
> A2 OK Success [THROTTLED]
> A3 LIST "" INBOX
> * LIST (\HasNoChildren) "/" "INBOX"
> A3 OK Success [THROTTLED]
> DEBUG IMAPS: connection available -- size: 1
> A4 SELECT INBOX
> * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $NotPhishing $Phishing)
> * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $NotPhishing 
> $Phishing \*)] Flags permitted.
> * OK [UIDVALIDITY 1] UIDs valid.
> * 78900 EXISTS
> * 0 RECENT
> * OK [UIDNEXT 78907] Predicted next UID.
> * OK [HIGHESTMODSEQ 3405665]
> A4 OK [READ-WRITE] INBOX selected. (Success) [THROTTLED]
> A5 SEARCH UNSEEN 1:78900
> * SEARCH 3 4 5 6 10 20 26 39 58 76 78 97 102 105 116 117 118 119 120 121 122 
> 123 124 125 ...  ... 78890 78891 78892 78893 
> 78894 78895 78896 78897 78898 78899 78900
> A5 OK SEARCH completed (Success) [THROTTLED]
> A6 FETCH 3 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
> * 3 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
> Message-ID: 
>
> )
> A6 OK Success [THROTTLED]
> A7 FETCH 4 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
> * 4 FETCH (BODY[HEADER.FIELDS (Message-ID)] {65}
> Message-Id: 
>
> )
> A7 OK Success [THROTTLED]
> A8 FETCH 5 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
> * 5 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
> Message-ID: 
>
> )
> A8 OK Success [THROTTLED]
> A9 FETCH 6 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
> * 6 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
> Message-ID: 
>
> )
> A9 OK Success [THROTTLED]
> A10 FETCH 10 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
> * 10 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
> Message-ID: 
>
> )
> A10 OK Success [THROTTLED]
> A11 FETCH 20 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
> * 20 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
> Message-ID: 
>
> )
> A11 OK Success 

imaps component not retrieving specified number of messages

2019-03-06 Thread js504
Hello,

I am trying to poll a gmail account to process emails using the camel-mail 
version 2.23.0 imaps component and the Java DSL.

There are thousands of emails I need to process but I am trying to split it up 
so as to not overload anything.

I have tried setting maxMessagesPerPoll and fetchSize parameters to different 
numbers but no matter what it seems to try to peek every message first ( I 
think ) and doesn't seem to retrieve just the number of messages I have 
specified.  I have tried setting peek to false as well but this does not seem 
to help.  I have also tried setting just the maxMessagesPerPoll without 
fetchSize and peek as well as setting just fetchSize without maxMessagesPerPoll 
or peek set but the same occurs.

below is my from statement using the imaps component:

from("imaps://{{env:IMAPS_HOSTNAME}}:{{env:IMAPS_PORT}}?" +
"maxMessagesPerPoll=10" +
"=10" +
"=false" +
"={{env:EMAIL_ADDRESS}}" +
"=" + password +
"={{env:IMAP_DEBUG_MODE}}" +
"=5000")

The following is the output from the connection to the gmail account and the 
following '(BODY.PEEK[HEADER.FIELDS (Message-ID)])'  messages that seem to go 
on and on for every message before retrieval of the message body has even begun.
Any explanation as to what is happening and what I could do in order to process 
say 1000 emails at a time and stop it from doing this pre-processing on all 
emails instead of the first 1000 per poll would be really appreciated.

2019-03-05 21:10:01,531  INFO [main] (DefaultCamelContext.java:3197) - Apache 
Camel 2.23.1 (CamelContext: consume-emails) started in 0.969 seconds
DEBUG: getProvider() returning 
javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle]
DEBUG IMAPS: mail.imap.fetchsize: 16384
DEBUG IMAPS: mail.imap.ignorebodystructuresize: false
DEBUG IMAPS: mail.imap.statuscachetimeout: 1000
DEBUG IMAPS: mail.imap.appendbuffersize: -1
DEBUG IMAPS: mail.imap.minidletime: 10
DEBUG IMAPS: closeFoldersOnStoreFailure
DEBUG IMAPS: trying to connect to host "imap.gmail.com", port 993, isSSL true
* OK Gimap ready for requests from 45.2.196.68 n3mb457716515jap
A0 CAPABILITY
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN 
X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN 
AUTH=OAUTHBEARER AUTH=XOAUTH
A0 OK Thats all she wrote! n3mb457716515jap
DEBUG IMAPS: AUTH: XOAUTH2
DEBUG IMAPS: AUTH: PLAIN
DEBUG IMAPS: AUTH: PLAIN-CLIENTTOKEN
DEBUG IMAPS: AUTH: OAUTHBEARER
DEBUG IMAPS: AUTH: XOAUTH
DEBUG IMAPS: protocolConnect login, host=imap.gmail.com, user=, 
password=
DEBUG IMAPS: AUTHENTICATE PLAIN command trace suppressed
DEBUG IMAPS: AUTHENTICATE PLAIN command result: A1 OK  
authenticated (Success)
A2 ENABLE UTF8=ACCEPT
* ENABLED UTF8=ACCEPT
A2 OK Success [THROTTLED]
A3 LIST "" INBOX
* LIST (\HasNoChildren) "/" "INBOX"
A3 OK Success [THROTTLED]
DEBUG IMAPS: connection available -- size: 1
A4 SELECT INBOX
* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $NotPhishing $Phishing)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $NotPhishing 
$Phishing \*)] Flags permitted.
* OK [UIDVALIDITY 1] UIDs valid.
* 78900 EXISTS
* 0 RECENT
* OK [UIDNEXT 78907] Predicted next UID.
* OK [HIGHESTMODSEQ 3405665]
A4 OK [READ-WRITE] INBOX selected. (Success) [THROTTLED]
A5 SEARCH UNSEEN 1:78900
* SEARCH 3 4 5 6 10 20 26 39 58 76 78 97 102 105 116 117 118 119 120 121 122 
123 124 125 ...  ... 78890 78891 78892 78893 
78894 78895 78896 78897 78898 78899 78900
A5 OK SEARCH completed (Success) [THROTTLED]
A6 FETCH 3 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 3 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A6 OK Success [THROTTLED]
A7 FETCH 4 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 4 FETCH (BODY[HEADER.FIELDS (Message-ID)] {65}
Message-Id: 

)
A7 OK Success [THROTTLED]
A8 FETCH 5 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 5 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A8 OK Success [THROTTLED]
A9 FETCH 6 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 6 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A9 OK Success [THROTTLED]
A10 FETCH 10 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 10 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A10 OK Success [THROTTLED]
A11 FETCH 20 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 20 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A11 OK Success [THROTTLED]
A12 FETCH 26 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 26 FETCH (BODY[HEADER.FIELDS (Message-ID)] {65}
Message-Id: 

)
A12 OK Success [THROTTLED]
A13 FETCH 39 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 39 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A13 OK Success [THROTTLED]
A14 FETCH 58 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 58 FETCH (BODY[HEADER.FIELDS (Message-ID)] {64}
Message-ID: 

)
A14 OK Success [THROTTLED]
A15 FETCH 76 (BODY.PEEK[HEADER.FIELDS (Message-ID)])
* 76 FETCH