I have narrowed down a performance bottleneck in the Processor I implemented
in a SEDA route using the AWS Kinesis component.
I tested against an "empty" Processor (i.e. that doesn't do anything) and
got the following metrics:
*For the "empty" Processor*:11:10| INFO | MarkerIgnoringBase.java 107 |
type=METER, name=kinesis-records-consumed, *count=140,
mean_rate=2.3331402604*
16133, m1=1.6458279659887582, m5=0.43339880218285814,
m15=0.15173246327279963, rate_unit=events/second
*For the "actual" Processor*: 11:15| INFO | MarkerIgnoringBase.java 107 |
type=METER, name=kinesis-records-consumed, *count=20,
mean_rate=0.3333031357470416*, m1=0.17909216046832732,
m5=0.058841979331650814, m15=0.02131567762219331, rate_unit=events/second
So, the bottleneck definitely seems to be in the "actual" Processor.
I have increased the number of shards on the streamas well as tweaked a host
of other parameters in order to improve performance with no real effect.
I now want to try using a custom ScheduledExecutorService to use multiple
threads for the consumer.
Is there an example of how to do this?
Also, concerning the number of records in the metrics above, how does the
Camel component determine how many records to read off the stream on the
first read? I'd expect them to be similar and maybe different on subsequent
reads?
--
View this message in context:
http://camel.465427.n5.nabble.com/AWS-Kinesis-Consumer-performance-improvement-tp5795628.html
Sent from the Camel - Users mailing list archive at Nabble.com.
