Yes. ActiveMQ is already there and uses Apache so nothing to do there. The libs
for AWS SQS also use the Apache license.
On 7/14/16, 2:42 PM, "Pramod Immaneni" <pra...@datatorrent.com> wrote:
What is the license covering the libraries, Apache?
On Thu, Jul 14, 2016 at 2:37 PM, Sanjay Pujare <san...@datatorrent.com>
wrote:
> Sounds good. Will go with test scope.
>
> On 7/14/16, 12:57 PM, "Ashwin Chandra Putta" <ashwinchand...@gmail.com>
> wrote:
>
> They can be made test scoped in malhar for testing. The user of this
> operator can add the implementation specific dependency in their
> application pom where they use this operator and supply the
> corresponding
> connection factory class name.
>
> Regards,
> Ashwin.
>
> On Thu, Jul 14, 2016 at 11:55 AM, Sanjay Pujare <
> san...@datatorrent.com>
> wrote:
>
> > Pramod,
> >
> > The JMS “wrapper” library needs the actual client library as you can
> see
> > in com.datatorrent.lib.io.jms.JMSBase.getConnectionFactory()
> >
> > Class<ConnectionFactory> clazz =
> > (Class<ConnectionFactory>)Class.forName(connectionFactoryClass);
> >
> > Where it loads the client library’s connectionFactoryClass by name.
> > Instead of putting the onus on the caller, I thought we can include
> these
> > libraries, so the current class loader can load the required class.
> >
> > Sanjay
> >
> >
> > On 7/14/16, 11:36 AM, "Pramod Immaneni" <pra...@datatorrent.com>
> wrote:
> >
> > Hi Sanjay,
> >
> > If there are going to separate operators developed for SQS and
> ActiveMQ
> > using the native api, why do we need to add the client libraries
> for
> > those
> > to JMS? Are these only needed for testing?
> >
> > Thanks
> >
> > On Thu, Jul 14, 2016 at 11:16 AM, Sanjay Pujare <
> > san...@datatorrent.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I am proposing the following enhancements to the Malhar JMS
> input
> > operator.
> > > Let me know if you have any comments.
> > >
> > > Enhancements are proposed to the malhar JMS Input Operator (in
> the
> > package
> > > com.datatorrent.lib.io.jms) to make it usable with any JMS
> compatible
> > > message broker for basic input functionality.
> > >
> > > For now we will make this input operator work with ActiveMQ
and
> > Amazon SQS
> > > through the JMS interface. The current code contains
> implementation
> > and
> > > tests for ActiveMQ, and we will add support and tests for SQS
> without
> > > breaking the ActiveMQ support.
> > >
> > > The motivation for this enhancement is as follows. JMS is NOT
> a wire
> > > protocol but just an API specification for Java programs, so
> JMS is
> > just a
> > > “wrapper” around an actual implementation library that talks
> to the
> > > respective message broker. Moreover each message broker has
> its own
> > > semantics for more advanced features like partitioning or
> dynamic
> > scaling
> > > that are impossible or difficult to capture via the JMS
> abstraction.
> > With
> > > this enhancement, we expect the JMS input operator to be
> usable as a
> > > generic “JMS input operator” with no support for advanced
> features
> > like
> > > partionability or dynamic scalability.
> > >
> > > Full featured input operators for SQS and ActiveMQ will be
> developed
> > > separately (without necessarily using the JMS interface but
> the most
> > > appropriate interface) and the design and implementation of
> those
> > operators
> > > are out of scope here.
> > >
> > > The enhancement includes:
> > >
> > > -
> > >
> > > adding amazon-sqs-java-messaging-lib and
aws-android-sdk-sqs
> > libraries
> > > to the pom
> > > -
> > >
> > > adding test cases for SQS, matching those for ActiveMQ.
> > > -
> > >
> > > making both sets of test cases work
> > > -
> > >
> > > cleaning up the existing code (e.g. removing hard coded
> values
> > like
> > > “TEST.FOO” in JMSBase)
> > >
> > > SQS ConsiderationsNo Support for Topics
> > >
> > > SQS only supports queues, so topics will not be supported for
> the SQS
> > > variant.
> > > Idempotency
> > >
> > > We will implement a simple idempotent design that uses the
> > > WindowDataManager to store the whole message and deletes the
> message
> > from
> > > SQS so we are not impacted by SQS’s visibility timeout
feature.
> > >
> > >
> > >
> > > Sanjay
> > >
> >
> >
> >
> >
>
>
> --
>
> Regards,
> Ashwin.
>
>
>
>
