Thank you for your messages. Let's try to gather as much information as
possible. We can then go ahead, summarize this in a JIRA issue and fix this for
the next release.
You say that the acknowledgement does not work at all. Do you have checkpointing
enabled? Flink will acknowledge messages as part of a checkpoint.
The connection error looks like a problem with the RabbitMQ connector itself.
An example application to reproduce this would be very helpful.
Thanks,
Max
On 11.12.18 11:44, Jeroen Steggink | knowsy wrote:
Hi,
We are experiencing the same thing. I tried RabbitMqIO 2.9.0-SNAPSHOT and
deployed on Flink 1.5.5.
The problems:
- RabbitMQ advices to share connections and have different channels. I don't
think that's possible with Beam when having both a Read and a Write RabbitMqIO?
- Flink job finishes without any error. I can't seem to figure out why. All the
messages are consumed, but not acknowledged.
- When I have a withQueueDeclare in the pipeline it only works the first time.
The second time I get a RabbitMq connection error and the pipeline is restarted
over and over again.
I tried making my own RabbitMq with the newer RabbitMq client, which is
asynchronous, but can't seem to make that work either.
All my other Java applications using the same RabbitMq client seem to be working
fine. Is it a threading issue maybe?
Cheers,
Jeroen
On 10-Dec-18 21:43, Schmukler, Stuart wrote:
Folks,
We have been trying to use RabbitMq with Beam via the RabbitIO
transformer(?). There have been problems connecting.
I’ll like to see if the RabbitIO 2.9.0 will work in our use case.
Make it a great day,
Stuart Schmukler
ZOOM: https://rakuten.zoom.us/j/9516474263
