Oleg, Thanks very much. Yes, I fully agree with you.
I saw your PR and noted that it’s for 1.x. Since it uses lambdas the patch is not going to work for the 0.x branch. I don’t know what your plan for 0.8 is but I have a much less sophisticated patch for the for the 0.x branch. Should I share? Chris McDermott Remote Business Analytics STaTS/StoreFront Remote HPE Storage Hewlett Packard Enterprise Mobile: +1 978-697-5315 On 9/15/16, 9:45 AM, "Oleg Zhurakousky" <[email protected]> wrote: Chris PR was submitted to the JIRA. Basically as I’ve commented in the JIRA I now do not believe we should be exposing any QOS options to the end user and instead implicitly ack messages only after they’ve been consumed by NiFi (i.e., transferred to a success relationship). Basically a classic chain of custody pattern. Let me know if you agree since adding ACK mode options would be a simply change, although at the moment I am reluctant to do so (see my explanation in JIRA) Cheers Oleg > On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <[email protected]> wrote: > > Will do. Thanks! > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <[email protected]> wrote: > > Chris > > The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is not being set explicitly it is set. > > We can definitely expose the other two (client ack and dups ok). > Please raise JIRA. > Cheers > Oleg > > On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <[email protected]<mailto:[email protected]>> wrote: > > Looking at the code the ACK mode is not being set. > > /** > * This method essentially performs initialization of this Processor by > * obtaining an instance of the {@link ConnectionFactory} from the > * {@link JMSConnectionFactoryProvider} (ControllerService) and performing a > * series of {@link ConnectionFactory} adaptations which eventually results > * in an instance of the {@link CachingConnectionFactory} used to construct > * {@link JmsTemplate} used by this Processor. > */ > private void buildTargetResource(ProcessContext context) { > if (this.targetResource == null) { > JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class); > ConnectionFactory connectionFactory = cfProvider.getConnectionFactory(); > > UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = new UserCredentialsConnectionFactoryAdapter(); > cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory); > cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue()); > cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue()); > > this.cachingConnectionFactory = new CachingConnectionFactory(cfCredentialsAdapter); > this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue())); > > JmsTemplate jmsTemplate = new JmsTemplate(); > jmsTemplate.setConnectionFactory(this.cachingConnectionFactory); > jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue()); > jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue())); > > // set of properties that may be good candidates for exposure via configuration > jmsTemplate.setReceiveTimeout(10000); > > this.targetResource = this.finishBuildingTargetResource(jmsTemplate); > } > > The JavaDoc for JMSTemplate says. > > Default settings for JMS Sessions are "not transacted" and "auto-acknowledge". As defined by the Java EE specification, the transaction and acknowledgement parameters are ignored when a JMS Session is created inside an active transaction, no matter if a JTA transaction or a Spring-managed transaction. To configure them for native JMS usage, specify appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" bean properties. > > If I understand this correctly it means that ConsumeJMS is always using auto-acknowledge which would explain my lost files on NiFi restart. > > Do you concur? > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <[email protected]<mailto:[email protected]>> wrote: > > Yes. > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 9:54 AM, "Oleg Zhurakousky" <[email protected]<mailto:[email protected]>> wrote: > > So you’re asking about ConsumeJMS? > > On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <[email protected]<mailto:[email protected]>> wrote: > > Hi Oleg, > > Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not expose a setting for ACK mode. Screen shot attached. So my question is which ACK mode does it use? > > Thanks, > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/14/16, 9:41 AM, "Oleg Zhurakousky" <[email protected]<mailto:[email protected]>> wrote: > > Chris > > FWIW, the Get/PutJMS* are effectively deprecated processors since they only support one JMS provider (ActiiveMQ) and should not be used. Instead we recommend Publish/ConsumeJMS pair. > > As to your question, which specific ACK mode you are inquiring about? As the doc currently states, the auto-ACK may loose messages. Is that what you are using? > > Cheers > Oleg > > On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <[email protected]<mailto:[email protected]><mailto:[email protected]>> wrote: > > > Hey folks, > > The GetJMSQueue processor allows configuration of the Acknowledgment Mode. > > Acknowledgement Mode > > Client Acknowledge > > • Client Acknowledge > • Auto Acknowledge > > The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge. > > > Can someone speak to how Acknowledgement Mode is handled by the ConsumeJMS processor? > > The reason I ask is that I seem to be losing messages on NiFi restart. > > Thanks, > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > > > > > > > > > > > >
