Chris PR was submitted to the JIRA. Basically as I’ve commented in the JIRA I now do not believe we should be exposing any QOS options to the end user and instead implicitly ack messages only after they’ve been consumed by NiFi (i.e., transferred to a success relationship). Basically a classic chain of custody pattern. Let me know if you agree since adding ACK mode options would be a simply change, although at the moment I am reluctant to do so (see my explanation in JIRA)
Cheers Oleg > On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote) <[email protected]> wrote: > > Will do. Thanks! > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <[email protected]> wrote: > > Chris > > The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is > not being set explicitly it is set. > > We can definitely expose the other two (client ack and dups ok). > Please raise JIRA. > Cheers > Oleg > > On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote) > <[email protected]<mailto:[email protected]>> wrote: > > Looking at the code the ACK mode is not being set. > > /** > * This method essentially performs initialization of this Processor by > * obtaining an instance of the {@link ConnectionFactory} from the > * {@link JMSConnectionFactoryProvider} (ControllerService) and > performing a > * series of {@link ConnectionFactory} adaptations which eventually > results > * in an instance of the {@link CachingConnectionFactory} used to > construct > * {@link JmsTemplate} used by this Processor. > */ > private void buildTargetResource(ProcessContext context) { > if (this.targetResource == null) { > JMSConnectionFactoryProviderDefinition cfProvider = > context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class); > ConnectionFactory connectionFactory = > cfProvider.getConnectionFactory(); > > UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = > new UserCredentialsConnectionFactoryAdapter(); > > cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory); > > cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue()); > > cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue()); > > this.cachingConnectionFactory = new > CachingConnectionFactory(cfCredentialsAdapter); > > this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue())); > > JmsTemplate jmsTemplate = new JmsTemplate(); > jmsTemplate.setConnectionFactory(this.cachingConnectionFactory); > > jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue()); > > jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue())); > > // set of properties that may be good candidates for exposure > via configuration > jmsTemplate.setReceiveTimeout(10000); > > this.targetResource = > this.finishBuildingTargetResource(jmsTemplate); > } > > The JavaDoc for JMSTemplate says. > > Default settings for JMS Sessions are "not transacted" and > "auto-acknowledge". As defined by the Java EE specification, the transaction > and acknowledgement parameters are ignored when a JMS Session is created > inside an active transaction, no matter if a JTA transaction or a > Spring-managed transaction. To configure them for native JMS usage, specify > appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" > bean properties. > > If I understand this correctly it means that ConsumeJMS is always using > auto-acknowledge which would explain my lost files on NiFi restart. > > Do you concur? > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote)" > <[email protected]<mailto:[email protected]>> wrote: > > Yes. > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 9:54 AM, "Oleg Zhurakousky" > <[email protected]<mailto:[email protected]>> wrote: > > So you’re asking about ConsumeJMS? > > On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote) > <[email protected]<mailto:[email protected]>> wrote: > > Hi Oleg, > > Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not > expose a setting for ACK mode. Screen shot attached. So my question is which > ACK mode does it use? > > Thanks, > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 9:41 AM, "Oleg Zhurakousky" > <[email protected]<mailto:[email protected]>> wrote: > > Chris > > FWIW, the Get/PutJMS* are effectively deprecated processors since they > only support one JMS provider (ActiiveMQ) and should not be used. Instead we > recommend Publish/ConsumeJMS pair. > > As to your question, which specific ACK mode you are inquiring about? As > the doc currently states, the auto-ACK may loose messages. Is that what you > are using? > > Cheers > Oleg > > On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote) > <[email protected]<mailto:[email protected]><mailto:[email protected]>> > wrote: > > > Hey folks, > > The GetJMSQueue processor allows configuration of the Acknowledgment > Mode. > > Acknowledgement Mode > > Client Acknowledge > > • Client Acknowledge > • Auto Acknowledge > > The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages > to be lost on restart of NiFi but may provide better performance than Client > Acknowledge. > > > Can someone speak to how Acknowledgement Mode is handled by the > ConsumeJMS processor? > > The reason I ask is that I seem to be losing messages on NiFi restart. > > Thanks, > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > > > > > > > > > > > >
