[
https://issues.apache.org/jira/browse/APEXMALHAR-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898492#comment-15898492
]
Sanjay M Pujare commented on APEXMALHAR-2434:
---------------------------------------------
Going with the last option ie. using deterministic naming for the metadata
queue, this is what we should do:
- for the actual application queue named "FOOBAR", the store should use the
metadata queue "FOOBAR.metadata".
- all operators (partitions) writing to "FOOBAR" use message properties
(javax.jms.Message.setIntProperty(String, int)) to set the operatorId with name
"operatorId" on the message to be written.
- these operators use message selectors in createBrowser(queue) and
createConsumers(queue) to only get messages with those respective message
properties e.g. operator 2 only gets messages with message property
operatorId==2.
This solution seems to be simple and elegant.
> JMSTransactionableStore uses Session.createQueue() which fails
> --------------------------------------------------------------
>
> Key: APEXMALHAR-2434
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2434
> Project: Apache Apex Malhar
> Issue Type: Bug
> Reporter: Sanjay M Pujare
>
> JMSTransactionableStore needs to create a queue for storing metadata
> (lastWindowId etc) that will work across invocations to support fault
> tolerance and idempotency.
> However as the createQueue Javadocs says: "Note that this method is not for
> creating the physical queue. The physical creation of queues is an
> administrative task and is not to be initiated by the JMS API." This causes a
> failure in actual tests with a production JMS based broker (such as IBM
> MQSeries). We will need to fix this in one of the following ways:
> - using an alternative store (HDFS or JDBC)
> - allow the user to specify a name for this metadata queue via a property
> - generate the name in a deterministic fashion from the subject of the queue
> The last 2 alternatives assume that the application user has created this
> metadata queue ahead of time from the admin console. We will need to document
> this in the malhar docs. The last alternative looks most attractive except if
> there are multiple JMS output operators (say partitions of an operator)
> writing to the same queue (subject) we will have to use some additional logic
> for them to share this single metadata queue.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
