Claus Ibsen created CAMEL-27192:
-----------------------------------

             Summary: camel-google-pubsub - publisher cache ignores ordering, 
no resumePublish
                 Key: CAMEL-27192
                 URL: https://issues.apache.org/jira/browse/CAMEL-27192
             Project: Camel
          Issue Type: Bug
          Components: camel-google-pubsub
            Reporter: Claus Ibsen


Two problems with message ordering in the producer, found by reading the code 
while adding {{messageOrderingEnabled}} to the {{google-pubsub-sink}} Kamelet.

h3. 1. The publisher cache is keyed by topic only

{{GooglePubsubComponent.getPublisher}} caches one {{Publisher}} per topic name:

{code:java}
return cachedPublishers.get(topicName, () -> buildPublisher(topicName, 
googlePubsubEndpoint));
{code}

but {{buildPublisher}} configures the publisher from the endpoint: 
{{messageOrderingEnabled}}, {{pubsubEndpoint}} and {{retry}}. A second endpoint 
publishing to the same topic with different settings gets the publisher built 
for the first one. For example, with

{code}
to("google-pubsub:my-project:orders")                                   // used 
first
to("google-pubsub:my-project:orders?messageOrderingEnabled=true&pubsubEndpoint=europe-west1-pubsub.googleapis.com:443")
{code}

the second endpoint publishes without ordering and to the global endpoint, and 
its messages that carry {{CamelGooglePubsubOrderingKey}} fail with {{Cannot 
publish a message with an ordering key when message ordering is not enabled}} 
(the error seen when ordering is off). Which endpoint wins depends on which one 
sends first after the cache entry expires ({{publisherCacheTimeout}}).

Suggested fix: key the cache by the topic plus the endpoint settings that 
{{buildPublisher}} uses (or cache per endpoint).

h3. 2. No resumePublish after a failed ordered publish

With ordering enabled, when a publish with an ordering key fails, the Google 
{{Publisher}} pauses that key: every later {{publish}} with the same key fails 
at once until {{Publisher.resumePublish(orderingKey)}} is called. 
{{GooglePubsubProducer.send}} calls {{publisher.publish(message)}} and 
{{messageIdFuture.get()}} but never {{resumePublish}} ({{resumePublish}} is not 
used anywhere in the component). One transient failure therefore blocks that 
ordering key for good: the error handler's redeliveries and all later messages 
with that key fail, until the cached publisher expires or the application 
restarts.

Suggested fix: in {{send}}, when {{messageIdFuture.get()}} throws and the 
message has an ordering key, call {{publisher.resumePublish(orderingKey)}} 
before rethrowing, so that a redelivery or the next message can be published. 
Document that ordering after a failure then depends on the route's error 
handling (e.g. redelivery before continuing).

h3. Where

* 
{{components/camel-google/camel-google-pubsub/src/main/java/org/apache/camel/component/google/pubsub/GooglePubsubComponent.java}}:
 {{getPublisher}}, {{buildPublisher}}.
* 
{{components/camel-google/camel-google-pubsub/src/main/java/org/apache/camel/component/google/pubsub/GooglePubsubProducer.java}}:
 {{send}}.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to