Good point about lambda. Let me see if I can refactor it a bit. And sure, please share your patch. Would love to compare.
Cheers Oleg > On Sep 15, 2016, at 11:10 AM, McDermott, Chris Kevin (MSDU - > STaTS/StorefrontRemote) <[email protected]> wrote: > > Oleg, > > Thanks very much. Yes, I fully agree with you. > > I saw your PR and noted that it’s for 1.x. Since it uses lambdas the patch > is not going to work for the 0.x branch. I don’t know what your plan for 0.8 > is but I have a much less sophisticated patch for the for the 0.x branch. > Should I share? > > Chris McDermott > > Remote Business Analytics > STaTS/StoreFront Remote > HPE Storage > Hewlett Packard Enterprise > Mobile: +1 978-697-5315 > > > > > On 9/15/16, 9:45 AM, "Oleg Zhurakousky" <[email protected]> wrote: > > Chris > > PR was submitted to the JIRA. Basically as I’ve commented in the JIRA I > now do not believe we should be exposing any QOS options to the end user and > instead implicitly ack messages only after they’ve been consumed by NiFi > (i.e., transferred to a success relationship). Basically a classic chain of > custody pattern. > Let me know if you agree since adding ACK mode options would be a simply > change, although at the moment I am reluctant to do so (see my explanation in > JIRA) > > Cheers > Oleg > >> On Sep 14, 2016, at 10:13 AM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) <[email protected]> wrote: >> >> Will do. Thanks! >> >> Chris McDermott >> >> Remote Business Analytics >> STaTS/StoreFront Remote >> HPE Storage >> Hewlett Packard Enterprise >> Mobile: +1 978-697-5315 >> >> >> >> >> On 9/14/16, 10:10 AM, "Oleg Zhurakousky" <[email protected]> >> wrote: >> >> Chris >> >> The “AUTO_ACKNOWLEDGE” (actual value 1) is the default. So although it is >> not being set explicitly it is set. >> >> We can definitely expose the other two (client ack and dups ok). >> Please raise JIRA. >> Cheers >> Oleg >> >> On Sep 14, 2016, at 10:04 AM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) >> <[email protected]<mailto:[email protected]>> wrote: >> >> Looking at the code the ACK mode is not being set. >> >> /** >> * This method essentially performs initialization of this Processor by >> * obtaining an instance of the {@link ConnectionFactory} from the >> * {@link JMSConnectionFactoryProvider} (ControllerService) and >> performing a >> * series of {@link ConnectionFactory} adaptations which eventually >> results >> * in an instance of the {@link CachingConnectionFactory} used to >> construct >> * {@link JmsTemplate} used by this Processor. >> */ >> private void buildTargetResource(ProcessContext context) { >> if (this.targetResource == null) { >> JMSConnectionFactoryProviderDefinition cfProvider = >> context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class); >> ConnectionFactory connectionFactory = >> cfProvider.getConnectionFactory(); >> >> UserCredentialsConnectionFactoryAdapter cfCredentialsAdapter = >> new UserCredentialsConnectionFactoryAdapter(); >> >> cfCredentialsAdapter.setTargetConnectionFactory(connectionFactory); >> >> cfCredentialsAdapter.setUsername(context.getProperty(USER).getValue()); >> >> cfCredentialsAdapter.setPassword(context.getProperty(PASSWORD).getValue()); >> >> this.cachingConnectionFactory = new >> CachingConnectionFactory(cfCredentialsAdapter); >> >> this.cachingConnectionFactory.setSessionCacheSize(Integer.parseInt(context.getProperty(SESSION_CACHE_SIZE).getValue())); >> >> JmsTemplate jmsTemplate = new JmsTemplate(); >> jmsTemplate.setConnectionFactory(this.cachingConnectionFactory); >> >> jmsTemplate.setDefaultDestinationName(context.getProperty(DESTINATION).getValue()); >> >> jmsTemplate.setPubSubDomain(TOPIC.equals(context.getProperty(DESTINATION_TYPE).getValue())); >> >> // set of properties that may be good candidates for exposure >> via configuration >> jmsTemplate.setReceiveTimeout(10000); >> >> this.targetResource = >> this.finishBuildingTargetResource(jmsTemplate); >> } >> >> The JavaDoc for JMSTemplate says. >> >> Default settings for JMS Sessions are "not transacted" and >> "auto-acknowledge". As defined by the Java EE specification, the transaction >> and acknowledgement parameters are ignored when a JMS Session is created >> inside an active transaction, no matter if a JTA transaction or a >> Spring-managed transaction. To configure them for native JMS usage, specify >> appropriate values for the "sessionTransacted" and "sessionAcknowledgeMode" >> bean properties. >> >> If I understand this correctly it means that ConsumeJMS is always using >> auto-acknowledge which would explain my lost files on NiFi restart. >> >> Do you concur? >> >> Chris McDermott >> >> Remote Business Analytics >> STaTS/StoreFront Remote >> HPE Storage >> Hewlett Packard Enterprise >> Mobile: +1 978-697-5315 >> >> >> >> >> On 9/14/16, 9:57 AM, "McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote)" >> <[email protected]<mailto:[email protected]>> wrote: >> >> Yes. >> >> Chris McDermott >> >> Remote Business Analytics >> STaTS/StoreFront Remote >> HPE Storage >> Hewlett Packard Enterprise >> Mobile: +1 978-697-5315 >> >> >> >> >> On 9/14/16, 9:54 AM, "Oleg Zhurakousky" >> <[email protected]<mailto:[email protected]>> wrote: >> >> So you’re asking about ConsumeJMS? >> >> On Sep 14, 2016, at 9:49 AM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) >> <[email protected]<mailto:[email protected]>> wrote: >> >> Hi Oleg, >> >> Sorry, I wasn’t more clear. I should have stated that ConsumeJMS does not >> expose a setting for ACK mode. Screen shot attached. So my question is >> which ACK mode does it use? >> >> Thanks, >> >> Chris McDermott >> >> Remote Business Analytics >> STaTS/StoreFront Remote >> HPE Storage >> Hewlett Packard Enterprise >> Mobile: +1 978-697-5315 >> >> >> >> >> On 9/14/16, 9:41 AM, "Oleg Zhurakousky" >> <[email protected]<mailto:[email protected]>> wrote: >> >> Chris >> >> FWIW, the Get/PutJMS* are effectively deprecated processors since they >> only support one JMS provider (ActiiveMQ) and should not be used. Instead we >> recommend Publish/ConsumeJMS pair. >> >> As to your question, which specific ACK mode you are inquiring about? As >> the doc currently states, the auto-ACK may loose messages. Is that what you >> are using? >> >> Cheers >> Oleg >> >> On Sep 14, 2016, at 9:28 AM, McDermott, Chris Kevin (MSDU - >> STaTS/StorefrontRemote) >> <[email protected]<mailto:[email protected]><mailto:[email protected]>> >> wrote: >> >> >> Hey folks, >> >> The GetJMSQueue processor allows configuration of the Acknowledgment >> Mode. >> >> Acknowledgement Mode >> >> Client Acknowledge >> >> • Client Acknowledge >> • Auto Acknowledge >> >> The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages >> to be lost on restart of NiFi but may provide better performance than Client >> Acknowledge. >> >> >> Can someone speak to how Acknowledgement Mode is handled by the >> ConsumeJMS processor? >> >> The reason I ask is that I seem to be losing messages on NiFi restart. >> >> Thanks, >> Chris McDermott >> >> Remote Business Analytics >> STaTS/StoreFront Remote >> HPE Storage >> Hewlett Packard Enterprise >> Mobile: +1 978-697-5315 >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > > > >
